AKPlayer crashes when playing from buffer on channelCount condition

user_2698072 0
Viewed 683

I struggle to make the following scenario work as expected (code will be provided below).

  1. Record my microphone input and store an AVAudioPCMBuffer in memory, this is done with AVAudioPCMBuffer extension method copy(from buffer: AVAudioPCMBuffer, readOffset: AVAudioFrameCount = default, frames: AVAudioFrameCount = default). I indeed get the buffer at the end of my recording.

  2. When record is ended pass the buffer to AKPlayer and play. Here is a code snippet to demonstrate what I do (I know it is no the full app code, if needed I can share it):

.

private var player: AKPlayer = AKPlayer()
self.player.buffering = .always

// in the record complete callbak:
self.player.buffer = self.bufferRecorder?.pcmBuffer
self.player.volume = 1
self.player.play()
  • please note that the plater is connected to a mixer which is eventually connected to the AudioKit output.

when I inspect and debug the application I could see the buffer is with the correct length, and all my output/input setup uses the same processing format (sample rate, channels, bitrate etc) as well as the buffer recorded, but still my app crashes on this line:

2018-10-28 08:40:32.625001+0200 BeatmanApp[71037:6731884] [avae] AVAEInternal.h:70:_AVAE_Check: 
required condition is false: [AVAudioPlayerNode.mm:665:ScheduleBuffer: (_outputFormat.channelCount == buffer.format.channelCount)]

when I debug and walk through the AudioKit code I can see that the breaking line is on AKPlayer+Playback.swift on line 162 on the method: playerNode.scheduleBuffer

more information that could be helpful:

  • the buffer recorded is 16 seconds long.
  • when I tried to pass the buffer straight to the player node in the tap method it seems as it worked, I did hear a delay from mic to speaker but it indeed played back.
  • I tried call prepare on the player before play method invoked, no help

thanks!

audiokit avfoundation avaudioplayer swift ios
1 Answers

Ok, this was super uncool debugging session. I had to investigate the AVAudioEngine and how this kind of scenario could be done there, which of course not the final result I was looking. This quest helped me to understand how to solve it with AudioKit (half of my app is implemented using AudioKit's tools so it doesn't make sense to rewrite it with AVFoundation).

AFFoundation solution:

private let engine = AVAudioEngine()
private let bufferSize = 1024
private let p: AVAudioPlayerNode = AVAudioPlayerNode()

let audioSession = AVAudioSession.sharedInstance()
do {
    try audioSession.setCategory(.playAndRecord, mode: .default, options: .defaultToSpeaker)
} catch {
    print("Setting category to AVAudioSessionCategoryPlayback failed.")
}

let inputNode = self.engine.inputNode
engine.connect(inputNode, to: engine.mainMixerNode, format: inputNode.inputFormat(forBus: 0))
// !!! the following lines are the key to the solution.
// !!! the player has to be attached to the engine before actually connected
engine.attach(p) 
engine.connect(p, to: engine.mainMixerNode, format: inputNode.inputFormat(forBus: 0)) 

do {
    try engine.start()
} catch {
    print("could not start engine \(error.localizedDescription)")
}

recordBufferAndPlay(duration: 4)

recordBufferAndPlay function:

func recordBufferAndPlay(duration: Double){
    let inputNode = self.engine.inputNode
    let total: Double = AVAudioSession.sharedInstance().sampleRate * duration
    let totalBufferSize: UInt32 = UInt32(total)

    let recordedBuffer : AVAudioPCMBuffer! = AVAudioPCMBuffer(pcmFormat: inputNode.inputFormat(forBus: 0), frameCapacity: totalBufferSize)

    var alreadyRecorded = 0
    inputNode.installTap(onBus: 0, bufferSize: 256, format: inputNode.inputFormat(forBus: 0)) {
        (buffer: AVAudioPCMBuffer!, time: AVAudioTime!) -> Void in
        recordedBuffer.copy(from: buffer) // this helper function is taken from audio kit!
        alreadyRecorded = alreadyRecorded + Int(buffer.frameLength)
        print(alreadyRecorded, totalBufferSize)
        if(alreadyRecorded >= totalBufferSize){
            inputNode.removeTap(onBus: 0)
            self.p.scheduleBuffer(recordedBuffer, at: nil, options: .loops, completionHandler: {
                print("completed playing")
            })
            self.p.play()
        }
    }
}

AudioKit solution:

So in the AudioKit solution these line should be invoked on your AKPlayer object. Note that this should be done before you actually start your engine.

self.player.buffering = .always
AudioKit.engine.attach(self.player.playerNode)
AudioKit.engine.connect(self.player.playerNode, to: self.mixer.inputNode, format: AudioKit.engine.inputNode.outputFormat(forBus: 0))

than the record is done pretty similarly to how you would have done it in AVAudioEngine, you install a tap on your node (microphone or other node) and record the buffer of PCM samples.

user_26980720
Related