OpenCV C++ memory leak when passing Mat to thread

Viewed 1200

I am trying to pass an OpenCV Mat from an image capture thread to an image processing thread as a part of a large application. I have no requirement for the image capture thread and the image processing thread to access the Mat at the same time. As a result, I simply want to pass ownership of the Mat from the image capture thread to the image processing thread. I am using OpenCV 3.

The problem I am encountering is that my program leaks a large amount of memory.

Below I have attached a minimal example of the code I am using to pass the Mats between threads. When I run this code it finishes using somewhere between 10MB and 500MB of memory from just 300 images with a resolution of 1640x1232.

Code

#include <iostream>
#include <thread>
#include <mutex>
#include <opencv2/opencv.hpp>
#include <signal.h>
#include <stdio.h>

#define SIM_PROCESSING_TIME_MS 150

using namespace std;
using namespace cv;

static volatile int keepRunning = 1;
static volatile int runThread = 1;

// Acts as a stack where the last image to be added is the first to be processed.
// If the 'capture' rate is higher than the processing rate then some images are skipped and processed at the end once the 'capture' has stopped.
vector<Mat> unprocessedImages;
mutex unprocessedImageMutex;

void intHandler(int dummy)
{
    keepRunning = 0;
}

// Simulates a function which captures an image using opencv.
Mat GetImage()
{
    return imread("../0000.jpg", CV_LOAD_IMAGE_COLOR);
}

// Simulates an image processing thread.
// A delay has been used to replace any actual image processing as in my testing it didn't seem to make a difference.
void image_processing_thread()
{
    int count = 0;
    while(true)
    {

        Mat imageToProcess;

        {// lock the vector and remove the last element
            lock_guard<mutex> lk(unprocessedImageMutex);
            if (unprocessedImages.size() > 0)
            {
                imageToProcess = unprocessedImages.back();
                unprocessedImages.pop_back();
            }
        }
        if(!imageToProcess.empty())
        {
            // We have an image to process so sleep to simulate processing
            this_thread::sleep_for(std::chrono::milliseconds(SIM_PROCESSING_TIME_MS));
            count++;
            cout << "Processed " << count << endl;
        }
        else if(!runThread) //The image loading thread is done and there are no more images to process
            break;

        this_thread::sleep_for(chrono::milliseconds(1));
    }
}

// Simulates the image capture thread.
// 'Captures' images then pushes them onto unprocessedImages which the image processing thread then reads from.
int main()
{
    signal(SIGINT, intHandler);

    // Start thread to process images
    auto imageProcessingThread = std::thread(image_processing_thread);

    // Load 300 images into memory
    for (int count = 0; count < 300; count++)
    {
        this_thread::sleep_for(std::chrono::milliseconds(20));
        auto img = GetImage();
        lock_guard<mutex> lk(unprocessedImageMutex);
        unprocessedImages.push_back(img);
    }

    // Allow processing thread to exit when it has finished
    runThread = 0;
    cout << "All images loaded in" << endl;
    imageProcessingThread.join();
    cout << "All images processed" << endl;

    while (keepRunning) {}
    return 1;
}

There is a bit of code so the program can be exited with a SIGINT. This code is not a part of my larger application

Attempted fixes

  • Calling unprocessedImages.reserve(1000) at the top of main.
  • Replacing unprocessedImages with a std::array and an index.
  • Replacing unprocessedImages with a c array and an index.
  • Using copyTo to move the Mats in an out of unprocessedImages.
  • All of the above while wrapping Mat in shared_ptr or unique_ptr (e.g. vector<unique_ptr<Mat>> unprocessedImages;).

None of these attempts had any affect on the characteristics of the memory leak.

Question

What is causing the memory leak in my program? How are you meant to pass ownership of OpenCV Mats between different threads?

Thanks, James

Edit: Added an additional attempted fix.

Edit 2: Running with Valgrind results in the above code NOT leaking. This is backed up by Valgrind's report which states that all blocks were reachable when the program ended. From the programs print outs it is clear that running it in valgrind made it a single threaded application as the print statements inside the two threads are perfectly interleaved.

Edit 3: I modified main like below. The result was that the peak and minimum memory usage on every iteration of the outer loop was the same. On this particular run the minimum memory usage was 374MB.

int main() {    
    signal(SIGINT, intHandler);

while(keepRunning)
{
runThread = 1;
    // Start thread to process images
    auto imageProcessingThread = std::thread(image_processing_thread);

    /* ... */

    cout << "All images processed" << endl;
}

    while (keepRunning) {}
    return 1;
}
0 Answers
Related