Acknowledgement Part 6

In the last update I detailed how I was replacing the entire audio stack, moving from the old AVAudioPlayer API to the newer AVAudioEngine based API. The main reason for doing this was to enable spakerphone mode. There doesn’t seem to be any way with the old API to play and record audio at the same time, without having the played audio being picked up by the recorder. With the new API, I can make one call:

try? avAudioEngine.inputNode.setVoiceProcessingEnabled(true)

and that enables echo cancellation.

And it works! It even works on the input tap. That is, I can install an input tap on the mic as normal:

    func installInputTap(block: @escaping AVAudioNodeTapBlock) {
        let inputNode = avAudioEngine.inputNode
        let micRecordingFormat = inputNode.outputFormat(forBus: 0)
        inputNode.installTap(onBus: 0, bufferSize: 1024, format: micRecordingFormat, block: block)
    }

And the bytes of audio that I receive are properly filtered so I don’t hear any clip that’s currently playing. That’s important for Reiterate, since with Acknowledge mode it’s listening for you to repeat the clip back, and if it hears itself play the clip then it gets a false trigger.

One other issue I had to deal with is playing audio files. Previously, all I had to do was instantiate an AVAudioPlayer and play my file, but the new API is a little more complex. You have to create an AVAudioPlayerNode, connect that node to your engine’s graph, then create an AVAudioFile from the file you want to play, and feed it the bytes via AVAudioPlayerNode.scheduleBuffer.

Another gotcha is file formats. The Apple documentation states:

Generally, you want to configure the node’s output format with the same number of channels as in the files and buffers. Otherwise, the node drops or adds channels as necessary.

I didn’t find this to be the case. I had specified dual-channel (stereo) format for my player node, but my clip files are recorded in mono. When I tried to play them, it raised an exception.

So, you do want to configure your player node to match the format of whatever file its playing. But what format is that? Apple’s audio file formats are quire confusing. The easiest solution I found was to simply set my node to whatever format it thought the audio file was:

   func configureClipPlayback(_ url: URL) {
        avAudioEngine.stop()
        avAudioEngine.disconnectNodeOutput(clipPlayer)
        guard let audioFile = try? AVAudioFile(forReading: url) else { return }
        avAudioEngine.connect(clipPlayer, to: avAudioEngine.mainMixerNode, format: audioFile.processingFormat)
    }

Of course, you can’t do this on the fly, so you have to stop your engine, disconnect the node, then reconnect it with the proper format. Fortunately, this is something that can be done once, since I’m not playing back multiple formats. If for some reason you do need to play multiple formats, I think it makes sense to have multiple player nodes.

I now have speakerphone acknowledge sent out for testing and am awaiting feedback. Still on my plate is reworking all the other audio code in the app to use the new stack. This will include playing existing clips outside a session, recording new clips, and the power meter display. But these should be minor details. The main feature is working.