Streaming audio and video in sync for mp4 container using Gstreamer framework

user_4653725 0
Viewed 10449

I am new to gstreamer and I want to stream a mp4 video which is having audio and video both from my Host(Ubuntu PC) to target board. I have successfully streamed only video. But streaming both(audio and video) is not happening.

Pipeline to stream only video is as below

Sender:

gst-launch-1.0 -v filesrc location = video.mp4 ! decodebin ! x264enc ! rtph264pay ! udpsink host=192.168.2.136 port=5000

receiver:

gst-launch-1.0 -v udpsrc port=5000 caps="application/x-rtp, media=video, clock-rate=90000, encoding-name=H264, payload=96, ssrc=3394826012, timestamp-offset=2215812541, seqnum-offset=46353" ! rtph264depay ! avdec_h264 ! videoconvert ! ximagesink sync=false

Hence, can see video on target.

Did streaming of only audio from mp4 file.

Pipeline to stream only video is as below

Sender:

gst-launch-1.0 -v filesrc location=video.mp4 ! qtdemux name=demuxer demuxer. ! queue ! rtpmp4apay pt=97 ! udpsink host=192.168.2.69 port=5000

Receiver:

gst-launch-1.0 -v udpsrc port=5000 caps="application/x-rtp, media=audio, clock-rate=44100, encoding-name=MP4A-LATM, cpresent=0, config=40002420, payload=97, ssrc=3386264266, timestamp-offset=2822202855, seqnum-offset=17719" ! rtpmp4adepay ! decodebin ! alsasink &

This too runs successfully.

But when I tried below pipeline to stream both audio/video in sync:

Sender:

gst-launch-1.0 -v filesrc location=sample.mp4 ! qtdemux name=demux demux. ! queue ! rtph264pay pt=96 ! udpsink host=192.168.3.101 port=5000 demux. ! queue ! rtpmp4apay pt=97 ! udpsink host=192.168.3.101 port=5001

Receiver:

gst-launch-1.0 -v udpsrc port=5000 caps="application/x-rtp, media=video, clock-rate=90000, encoding-name=H264, payload=96, ssrc=2226494377, timestamp-offset=3242004369, seqnum-offset=17021" ! rtph264depay ! decodebin ! ximagesink udpsrc port=5001 caps="application/x-rtp, media=audio, clock-rate=44100, encoding-name=MP4A-LATM, cpresent=0, config=40002420, payload=97, ssrc=1499706502, timestamp-offset=190741668, seqnum-offset=24774" ! rtpmp4adepay ! decodebin ! alsasink

But: get an output as below:

ERROR: from element /GstPipeline:pipeline0/GstDecodeBin:decodebin1/GstFaad:faad0: Could not decode stream.

fails. Please can someone suggest me a pipeline which can stream audio/video both on target device in sync.

Thanks in advance.

libav rtp video-streaming gstreamer
3 Answers

Sender

gst-launch-1.0 filesrc location= ~/1.mp4 ! qtdemux name=d ! queue ! h264parse ! rtph264pay config-interval=10 pt=96 ! udpsink host=10.24.217.30 port=5000 d. ! queue ! aacparse ! rtpmp4apay pt=97 ! udpsink host=10.24.217.30 port=5001 -v

Receiver

gst-launch-1.0 udpsrc port=5000 caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, payload=(int)96, encoding-name=(string)H264" ! rtph264depay ! identity silent=0 ! avdec_h264 ! videoconvert ! ximagesink sync=0 async=0 udpsrc port=5001 caps="application/x-rtp, media=audio, clock-rate=44100, encoding-name=MP4A-LATM, cpresent=0, config=40002420, payload=97, ssrc=1499706502, timestamp-offset=190741668, seqnum-offset=24774" ! rtpmp4adepay ! avdec_aac ! identity silent=0 ! alsasink sync=0 async=0

This works properly.

Note: Starting receiver early before starting sender gives error at the receiver end when sender is started. ** (gst-launch-1.0:26959): CRITICAL **: 15:15:51.191: gst_audio_decoder_set_output_format: assertion 'GST_AUDIO_INFO_IS_VALID (info)' failed ERROR: from element /GstPipeline:pipeline0/avdec_aac:avdec_aac0: GStreamer error: negotiation problem. Additional debug info: gstavauddec.c(449): gst_ffmpegauddec_negotiate (): /GstPipeline:pipeline0/avdec_aac:avdec_aac0: Could not set caps for libav decoder (aac), not fixed?

This could be a bug in GST in caps negotiations. WAR is to start receiver after sender.

user_61478900
Related