CMBlockBuffer data retention in Swift

user_4259255 0
Viewed 104

How should memory ownership be handled for a CMBlockBuffer in Swift?

Here's an example - displaying an incoming H.264 stream in AVSampleBufferDisplayLayer:

// 1. Copy incoming data
var dataCopy = dataFromStream
var count = dataCopy.count

// 2. Store data somewhere globally - ** crashes otherwise **
globalStorage.append(dataCopy) // globalStorage: [Data]

// 3. Create CMBLockBuffer
var blockBuffer: CMBlockBuffer!
CMBlockBufferCreateWithMemoryBlock(
    allocator: kCFAllocatorDefault,
    memoryBlock: UnsafeMutableRawPointer(mutating: (dataCopy as NSData).bytes),
    blockLength: count,
    blockAllocator: kCFAllocatorDefault,
    customBlockSource: nil,
    offsetToData: 0,
    dataLength: count,
    flags: 0,
    blockBufferOut: &blockBuffer)

// 4. Create CMSampleBuffer
var sampleBuffer: CMSampleBuffer!
CMSampleBufferCreate(
    allocator: kCFAllocatorDefault,
    dataBuffer: blockBuffer,
    dataReady: true,
    makeDataReadyCallback: nil,
    refcon: nil,
    formatDescription: myFormatDescription,
    sampleCount: 1,
    sampleTimingEntryCount: 0,
    sampleTimingArray: nil,
    sampleSizeEntryCount: 1,
    sampleSizeArray: &count,
    sampleBufferOut: &sampleBuffer)

// 5. Send to AVSampleBufferDisplayLayer...
myVideoLayer.enqueue(sampleBuffer)

If I remove step 2 from the above - i.e. the underlying data is not retained externally - I get the following crash after a handful of frames:

malloc: double free for ptr 0x1490efc00

Obviously, I can't keep appending data to a buffer forever. But if I set a cap on globalStorage and free old frames long after they've been displayed, I get a similar crash as soon as any array element is removed:

malloc: *** error for object 0x132619070: pointer being freed was not allocated

How are we supposed to work with CMBlockBuffer in Swift?

Edit

Using kCFAllocatorNull as the blockAllocator param when creating the buffer does prevent the crash.

However, we still need to keep the underlying data around until after the frames have been displayed. A circular buffer would work here, but the size needed can depend on a number of factors and change over time...

avfoundation h.264 swift objective-c
1 Answers

Solution

Replace steps 1 and 2 above with this:

let ptr = UnsafeMutableRawBufferPointer.allocate(byteCount: count, alignment: MemoryLayout<Data>.alignment)
ptr.copyBytes(from: dataFromStream)
// Then use ptr when creating CMBlockBuffer

Normally we'd be in charge of freeing this memory manually, but in this case if we keep kCFAllocatorDefault as our blockAllocator, the memory will be released automatically with the CMBlockBuffer (after the video layer has presented and discarded the containing CMSampleBuffer).

user_42592550
Related