I'm trying to send images with certain metadata over UDP using gstreamer. The sender process grabs images from camera, and sends them in the pipeline as appsrc ! rtpvrawpay ! udpsink (the full pipeline string below, in case it's important). And on the reciever side I use udpsrc ! rtpvrawdepay ! appsink.
In order to add metadata I put a probe on the Source Pad of rtpvrawpay, and to read - probe on Sink Pad of rtpvrawdepay. The problems I'm having so far:
- The "source" probe receives
GST_PAD_PROBE_TYPE_BUFFER_LISTdata, while the "sink" probe -GST_PAD_PROBE_TYPE_BUFFER. I would expect that I will get the same data type on both sides of UDP, so why it's not? - I fail to read the
twobytes_headerthat was added by the sender, it fails already when trying togst_rtp_buffer_mapthe buffer in the probe. UPDATE: succeeded to read the header on the receiver side, see the code below. Still would like to understand why I see Buffer List VS Buffer on the two probe locations.
Will appreciate any insights, or maybe a better direction for sending metadata with image over UDP.
Sender pipeline: appsrc ! videoconvert ! queue ! rtpvrawpay name=ToRTP ! queue ! udpsink host=192.168.42.49 port=5566 sync=false async=false
Reciever pipeline: udpsrc port=5566 caps="application/x-rtp, media=(string)video, sampling=(string)BGR, clock-rate=(int)90000, depth=(string)8, width=(string)640, height=(string)480, encoding-name=(string)RAW" ! rtpvrawdepay name=FromRTP ! queue ! videoconvert ! appsink
UPDATE: The Code that worked for me:
Sender Code:
GstRTPBuffer rtp_buf = GST_RTP_BUFFER_INIT;
gst_rtp_buffer_map(*buffer, GST_MAP_READWRITE, &rtp_buf);
gst_rtp_buffer_add_extension_twobytes_header(&rtp_buf, 1, 1, myInfo->buf, sizeof(myInfo->buf));
gst_rtp_buffer_unmap(&rtp_buf);
Receiver Code:
GstRTPBuffer rtp_buf = GST_RTP_BUFFER_INIT;
GstBuffer* buf = GST_PAD_PROBE_INFO_BUFFER(info);
gst_rtp_buffer_map(buf, GST_MAP_READ, &rtp_buf);
gpointer myInfoBuf = nullptr;
guint size = 0;
gst_rtp_buffer_get_extension_twobytes_header(&rtp_buf, &appbits, 1, 0, &myInfoBuf, &size);
gst_rtp_buffer_unmap(&rtp_buf);