Video broadcast using NDI SDK 4.5 in iOS 13 not working. Receiver in LAN does not receive any video packets

user_638898 0
Viewed 1042

I have been trying to use NDI SDK 4.5, in a Objective-C iOS-13 app, to broadcast camera capture from iPhone device.

My sample code is in public Github repo: https://github.com/bharatbiswal/CameraExampleObjectiveC

Following is how I send CMSampleBufferRef sampleBuffer:

CVPixelBufferRef pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);

NDIlib_video_frame_v2_t video_frame;
video_frame.xres = VIDEO_CAPTURE_WIDTH;
video_frame.yres = VIDEO_CAPTURE_HEIGHT;
video_frame.FourCC = NDIlib_FourCC_type_UYVY; // kCVPixelFormatType_420YpCbCr8BiPlanarFullRange
video_frame.line_stride_in_bytes = VIDEO_CAPTURE_WIDTH * VIDEO_CAPTURE_PIXEL_SIZE;
video_frame.p_data = CVPixelBufferGetBaseAddress(pixelBuffer);
NDIlib_send_send_video_v2(self.my_ndi_send, &video_frame);

I have been using "NewTek NDI Video Monitor" to receive the video from network. However, even though it shows as source, the video does not play.

Has anyone used NDI SDK in iOS to build broadcast sender or receiver functionalities? Please help.

xcode11 ios13
3 Answers

You should use kCVPixelFormatType_32BGRA in video settings. And NDIlib_FourCC_type_BGRA as FourCC in NDIlib_video_frame_v2_t.

user_91597650

Are you sure about your VIDEO_CAPTURE_PIXEL_SIZE ?

When I worked with NDI on macos I had the same black screen problem and it was due to a wrong line stride.

Maybe this can help : https://developer.apple.com/documentation/corevideo/1456964-cvpixelbuffergetbytesperrow?language=objc ?

Also it seems the pixel formats from core video and NDI don't match.

On the core video side you are using Bi-Planar Y'CbCr 8-bit 4:2:0, and on the NDI side you are using NDIlib_FourCC_type_UYVY which is Y'CbCr 4:2:2.

I cannot find any Bi-Planar Y'CbCr 8-bit 4:2:0 pixel format on the NDI side.

You may have more luck using the following combination:

Hope this helps!

user_5725430

In my experience, you have two mistake. To use CVPixelBuffer's CVPixelBufferGetBaseAddress, the CVPixelBufferLockBaseAddress method must be called first. Otherwise, it returns a null pointer.

https://developer.apple.com/documentation/corevideo/1457128-cvpixelbufferlockbaseaddress?language=objc

Secondly, NDI does not support YUV420 biplanar. (The default format for iOS cameras.) More precisely, NDI only accepts one data pointer. In other words, you have to merge the biplanar memory areas into one, and then pass it in NV12 format. See the NDI document for details.

So your code should look like this: And if sending asynchronously instead of NDIlib_send_send_video_v2, a strong reference to the transferred memory area must be maintained until the transfer operation by the NDI library is completed.

CVPixelBufferRef pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
CVPixelBufferLockBaseAddress(pixelBuffer, 0);
int width = (int)CVPixelBufferGetWidth(pixelBuffer);
int height = (int)CVPixelBufferGetHeight(pixelBuffer);
OSType pixelFormat = CVPixelBufferGetPixelFormatType(pixelBuffer);

NDIlib_FourCC_video_type_e ndiVideoFormat;
uint8_t* pixelData;
int stride;

if (pixelFormat == kCVPixelFormatType_32BGRA) {
    ndiVideoFormat = NDIlib_FourCC_type_BGRA;
    pixelData = (uint8_t*)CVPixelBufferGetBaseAddress(pixelBuffer); // Or copy for asynchronous transmit.
    stride = width * 4;        
} else if (pixelFormat == kCVPixelFormatType_420YpCbCr8BiPlanarFullRange) {
    ndiVideoFormat = NDIlib_FourCC_type_NV12;
    pixelData = (uint8_t*)malloc(width * height * 1.5);
    uint8_t* yPlane = (uint8_t*)CVPixelBufferGetBaseAddressOfPlane(pixelBuffer, 0);
    int yPlaneBytesPerRow = (int)CVPixelBufferGetBytesPerRowOfPlane(pixelBuffer, 0);
    int ySize = yPlaneBytesPerRow * height;
    uint8_t* uvPlane = (uint8_t*)CVPixelBufferGetBaseAddressOfPlane(pixelBuffer, 1);
    int uvPlaneBytesPerRow = (int)CVPixelBufferGetBytesPerRowOfPlane(pixelBuffer, 1);
    int uvSize = uvPlaneBytesPerRow * height;
    stride = yPlaneBytesPerRow;

    memcpy(pixelData, yPlane, ySize);
    memcpy(pixelData + ySize, uvPlane, uvSize);
} else {
    return;
}

NDIlib_video_frame_v2_t video_frame;
video_frame.xres = width;
video_frame.yres = height;
video_frame.FourCC = ndiVideoFormat;
video_frame.line_stride_in_bytes = stride;
video_frame.p_data = pixelData;
NDIlib_send_send_video_v2(self.my_ndi_send, &video_frame); // synchronous sending.

CVPixelBufferUnlockBaseAddress(pixelBuffer, 0);

// For synchrnous sending case. Free data or use pre-allocated memory.
if (pixelFormat == kCVPixelFormatType_420YpCbCr8BiPlanarFullRange) {
    free(pixelData);
}
user_129591730
Related