How to create a video stream when I have raw H264 frames and timestamps?

user_1031014 0
Viewed 1719

I receive from a third party application, a stream of packests containing raw H264 frames and a timestamp. I want to take the raw frames and the timestamp and re-stream, so I can see the video in a player like VLC

I tried to just take the raw H264 frames as they arrive and stream them over a TCP socket, and I'm actually able to see the video in VLC. But of course without timestamps, the video play at maximum speed and then it stop to wait other frames, and then it starts again at max speed. I guess I need to put those raw frames in a container and mark each frame with the timestamp, but I don't know how. Any help appreceated (if in C# even better)

Thanks

video-processing video-streaming h.264 c#
2 Answers

(1) Option 1: Timer

Simply use a timer to send your individual H264 frames. 

timer_delay = (1000/FPS); //if FPS is 30, then sends a frame every 33 milliseconds.

A frame (NAL unit) has a start code of four bytes as 00 00 00 00 so grab all bytes from one sequence of and including the 00 00 00 00 up to the last byte before another sequence of 00 00 00 00 begins. This should* be your frame data to send.

note:
I say "should" because H264 sometimes slices the images, so if you send a frame and get only half a picture that means you must send two or three sequences of NAL units as one whole item (eg: in one bytes array) to see a full image. You'll learn through active experimenting.

(2) Option 2: Pipes
Try FFmpeg a free command line tool. Actually it's also the audio/video decoder for VLC. You can use C#'s Standard in/out to send the received bytes to FFmpeg (which as a Process is running with encoder settings eg: the output format and resolution, etc).

See this article as a starting point:
https://mathewsachin.github.io/blog/2017/07/28/ffmpeg-pipe-csharp.html

user_20577090

Frames can be three or four byte delimiter (long or short) depending on encoder. However, each sample (i.e , collection on IDR and Non-IDR frames) have start sequence of 4 bytes 00 00 00 01. Refer to RFC documents for detail. You can collect frames in to a single sample and can send it over RTP/RTSP. Gstreamer has good capability for RTP.

user_70969210
Related