SourceBuffer.remove(start, end) removes whole buffered TimeRange (How to handle realtime stream with MSE?)

user_883777 0
Viewed 1997

I have a SourceBuffer with a single entry in .buffered. I have a realtime stream of raw h.264 data arriving which I encode into mp4 and push into the SourceBuffer with .appendBuffer(data). Since this is a realtime stream of data I need to keep clearing the buffer however this is where I encounter my problem. (Ie. I encounter a QuotaExceededError)

For examples sake my single entry in SourceBuffer.buffered has a timerange of 0-10 seconds. My attempt to tidy the buffer is to call SourceBuffer.remove(0, 8). My expectation is that my buffer would be cleared and I'd be left with a timerange of 8-10. However the entire timerange (my only range) is removed and from this point all further appendBuffer calls seem to do nothing.

Three questions relevant to this issue:

  1. How do I a) stop .remove from having this behaviour or b) force new time-ranges in my buffer so that only "old" ranges are removed.
  2. Why do the later appendBuffer calls do nothing? I would expect them to re-populate the SourceBuffer.
  3. Is there a better "MSE" way to handle a realtime stream where I never care about going back in time? Ie. All rendered data can be thrown away.

In case there's some weird Browser/Platform issue going on I'm using Chrome on Ubuntu.

Also, I am basing my code off of https://github.com/xevokk/h264-converter.

media-source video-streaming html5-video javascript
1 Answers

It's all in the MSE spec.

http://w3c.github.io/media-source/#sourcebuffer-coded-frame-removal

Step 3.3: Remove all media data, from this track buffer, that contain starting timestamps greater than or equal to start and less than the remove end timestamp.

So the user agent will remove all the data you've requested, from 0 to 8s

Then Step 3.4: Remove all possible decoding dependencies on the coded frames removed in the previous step by removing all coded frames from this track buffer between those frames removed in the previous step and the next random access point after those removed frames.

The user agent will remove all frames that depend on the ones you've just removed. Due to the way h264 works (and all modern video codec) that is all frames following the last keyframe until the next keyframe, as none of those frames can now be decoded.

There is no keyframe in range 8 to 10s, so they are all removed

Why do the later appendBuffer calls do nothing? I would expect them to re-populate the SourceBuffer.

You have removed data, as per spec, the next frame you add must be a keyframe. If the segment you add contains no keyframe, nothing will be added.

If the data you add is made of a single keyframe at the start followed by just P-frame, then you can't remove any frames in the middle without rendering unusable all the ones that follow

user_2986990
Related