Handling available input buffers in async MediaCodec for decoding when no new data is available

user_14635103 0
Viewed 442

I am decoding H.264 video data received through a websocket with an async Android MediaCodec.

My question is how to handle input buffers when there is no new encoded data to decode?

Currently I queue the input buffers without any data when there is no new encoded data from the websocket:

override fun onInputBufferAvailable(codec: MediaCodec, index: Int) {
    if (videoFrames.isNotEmpty()) {
        val inputBuffer = codec.getInputBuffer(index)
        inputBuffer?.put(videoFrames.removeFirst())
        codec.queueInputBuffer(index, 0, currentFrame.size, 0, 0)
    } else {
        codec.queueInputBuffer(index, 0, 0, 0, 0)
    }
}

where videoFrames is a list of frames with encoded video data.

I do it like above because if I alternatively remove the else clause and don't queue the empty input buffer, the input buffer never becomes available again.

Is there a different way of handling this? It feels wrong to me to queue an empty input buffer.

android-mediacodec android
1 Answers

An alternative solution that seemed to increase the decoder's performance by a bit was to store the available input buffers in a queue and fetch them when new encoded video data is received. Something like the following:

val videoFrames = ConcurrentList<ByteArray>()
val availableBuffers = mutableListOf<Int>()
var decoder: MediaCodec? = null

fun onInputBufferAvailable(codec: MediaCodec, index: Int) {
    availableBuffers.add(index)
}

fun processFrame(frame: ByteArray) {
    var frameToProcess = frame

    if (videoFrames.isNotEmpty()) {
        videoFrames.add(frame)
        frameToProcess = videoFrames.removeFirst()
    }

    if (availableBuffers.isNotEmpty() && decoder != null) {
        val bufferIndex = availableBuffers.removeFirst()
        val inputBuffer = decoder!!.getInputBuffer(bufferIndex)

        inputBuffer?.put(frameToProcess)
        decoder!!.queueInputBuffer(bufferIndex, 0, frameToProcess.size, 0, 0)
    }
}
user_146351030
Related