Streaming RTSP from multiple images from memory in C++ using ffmpeg

Viewed 387

I'm writing an application that shows text on screen. I would like to be able to stream the screen (with the text) via RTSP, so that a program like OBS/vMix or VLC could capture it. What I'm trying to do is to take a screenshot every N milliseconds (say every 100 milliseconds), save it into memory (not disk), then write it into a pipe that goes to ffmpeg. Then ffmpeg takes those images and makes a RTSP stream out of them. I've implemented the part where I take a screenshot every N milliseconds, and I write it into the pipe (similar to this question, the difference is that I save the image in memory, not on disk). The problem is now that I don't know much about ffmpeg (I've set the parameters as in the answer of this question, but I see nothing on screen when I try to play the stream via VLC or other program). I can see a lot of Successfully sent a screenshot in the logs, but nothing in the actual stream. Can you tell me what I'm doing wrong?

You can find the entire project here

Here's (some of) my code:

bool RTSP_SERVER_SHOULD_START = false;
tthread::mutex rtsp_server_mutex;
tthread::mutex screenshot_mutex;
tthread::thread* rtsp;
FILE *pPipe;
bool send_screenshot = false;

int main(int argc, char *argv[]) {
// OpenGL stuff...
        rtsp_server_mutex.lock();
        if (RTSP_SERVER_SHOULD_START) {
            RTSP_SERVER_SHOULD_START = false;
            rtsp = new tthread::thread(rtspScreenShotToPipe, 0);
            if( (pPipe = _popen( "ffmpeg -re -f image2pipe -vcodec mjpeg -i - -vcodec h264 -r 10 -f mpegts udp://127.0.0.1:1234", "wb")) == NULL ) {
                cerr << "Error: Could not open ffmpeg" << endl;
                _pclose(pPipe);
                exit(1);
            }
        }
        rtsp_server_mutex.unlock();

    while (!glfwWindowShouldClose(WINDOW)) {
        // More OpenGL stuff
        screenshot_mutex.lock();
        if(send_screenshot) {
            send_screenshot = false;
            // Make the BYTE array, factor of 3 because it's RBG.
            BYTE* pixels = new BYTE[3 * DEFAULT_MONITOR.maxResolution.width * DEFAULT_MONITOR.maxResolution.height];

            glReadPixels(0, 0, DEFAULT_MONITOR.maxResolution.width, DEFAULT_MONITOR.maxResolution.height, GL_RGB, GL_UNSIGNED_BYTE, pixels);

            // Convert to FreeImage format & save to file
            FIBITMAP* image = FreeImage_ConvertFromRawBits(pixels, DEFAULT_MONITOR.maxResolution.width, DEFAULT_MONITOR.maxResolution.height, 3 * DEFAULT_MONITOR.maxResolution.width, 24, 0x0000FF, 0xFF0000, 0x00FF00, false);

            FreeImageIO io;
            SetDefaultIO(&io);

            bool success = FreeImage_SaveToHandle(FIF_JPEG, image, &io, (fi_handle)pPipe);
            fflush(pPipe);
            if(success) {
                cout << "Successfully sent a screenshot" << endl;
            }

            // Free resources
            FreeImage_Unload(image);
            delete [] pixels;
        }
        screenshot_mutex.unlock();
    }
    printf("%d", glGetError());

    glfwTerminate();
    return 0;
}

void rtspScreenShotToPipe(void * aArg) {
    while(true) {
        rtsp_server_mutex.try_lock();
        if(!RTSP_SERVER_STARTED) {
            break;
        }
        rtsp_server_mutex.unlock();
        tthread::this_thread::sleep_for(tthread::chrono::milliseconds(100));
        screenshot_mutex.try_lock();
        send_screenshot = true;
        screenshot_mutex.unlock();
    }
}

// I've copied those function from FreeImage in my code

void SetDefaultIO(FreeImageIO *io) {
    io->read_proc  = _ReadProc;
    io->seek_proc  = _SeekProc;
    io->tell_proc  = _TellProc;
    io->write_proc = _WriteProc;
}

unsigned DLL_CALLCONV _ReadProc(void *buffer, unsigned size, unsigned count, fi_handle handle) {
    return (unsigned)fread(buffer, size, count, (FILE *)handle);
}

unsigned DLL_CALLCONV _WriteProc(void *buffer, unsigned size, unsigned count, fi_handle handle) {
    return (unsigned)fwrite(buffer, size, count, (FILE *)handle);
}

int DLL_CALLCONV _SeekProc(fi_handle handle, long offset, int origin) {
    return fseek((FILE *)handle, offset, origin);
}

long DLL_CALLCONV _TellProc(fi_handle handle) {
    return ftell((FILE *)handle);
}

1 Answers

You're not using RTSP, you're sending MPEG-TS over UDP.

In VLC make sure you listen for the correct network stream:

udp://@127.0.0.1:1234

On the FFmpeg side you need to specify the payload size:

-f mpegts udp://127.0.0.1:1234?pkt_size=1316

FFmpeg has an API that can be used directly from C++, so you don't have to pipe to a process.

Related