I am trying to pass an OpenCV Mat from an image capture thread to an image processing thread as a part of a large application. I have no requirement for the image capture thread and the image processing thread to access the Mat at the same time. As a result, I simply want to pass ownership of the Mat from the image capture thread to the image processing thread. I am using OpenCV 3.
The problem I am encountering is that my program leaks a large amount of memory.
Below I have attached a minimal example of the code I am using to pass the Mats between threads. When I run this code it finishes using somewhere between 10MB and 500MB of memory from just 300 images with a resolution of 1640x1232.
Code
#include <iostream>
#include <thread>
#include <mutex>
#include <opencv2/opencv.hpp>
#include <signal.h>
#include <stdio.h>
#define SIM_PROCESSING_TIME_MS 150
using namespace std;
using namespace cv;
static volatile int keepRunning = 1;
static volatile int runThread = 1;
// Acts as a stack where the last image to be added is the first to be processed.
// If the 'capture' rate is higher than the processing rate then some images are skipped and processed at the end once the 'capture' has stopped.
vector<Mat> unprocessedImages;
mutex unprocessedImageMutex;
void intHandler(int dummy)
{
keepRunning = 0;
}
// Simulates a function which captures an image using opencv.
Mat GetImage()
{
return imread("../0000.jpg", CV_LOAD_IMAGE_COLOR);
}
// Simulates an image processing thread.
// A delay has been used to replace any actual image processing as in my testing it didn't seem to make a difference.
void image_processing_thread()
{
int count = 0;
while(true)
{
Mat imageToProcess;
{// lock the vector and remove the last element
lock_guard<mutex> lk(unprocessedImageMutex);
if (unprocessedImages.size() > 0)
{
imageToProcess = unprocessedImages.back();
unprocessedImages.pop_back();
}
}
if(!imageToProcess.empty())
{
// We have an image to process so sleep to simulate processing
this_thread::sleep_for(std::chrono::milliseconds(SIM_PROCESSING_TIME_MS));
count++;
cout << "Processed " << count << endl;
}
else if(!runThread) //The image loading thread is done and there are no more images to process
break;
this_thread::sleep_for(chrono::milliseconds(1));
}
}
// Simulates the image capture thread.
// 'Captures' images then pushes them onto unprocessedImages which the image processing thread then reads from.
int main()
{
signal(SIGINT, intHandler);
// Start thread to process images
auto imageProcessingThread = std::thread(image_processing_thread);
// Load 300 images into memory
for (int count = 0; count < 300; count++)
{
this_thread::sleep_for(std::chrono::milliseconds(20));
auto img = GetImage();
lock_guard<mutex> lk(unprocessedImageMutex);
unprocessedImages.push_back(img);
}
// Allow processing thread to exit when it has finished
runThread = 0;
cout << "All images loaded in" << endl;
imageProcessingThread.join();
cout << "All images processed" << endl;
while (keepRunning) {}
return 1;
}
There is a bit of code so the program can be exited with a SIGINT. This code is not a part of my larger application
Attempted fixes
- Calling
unprocessedImages.reserve(1000)at the top of main. - Replacing
unprocessedImageswith astd::arrayand an index. - Replacing
unprocessedImageswith a c array and an index. - Using
copyToto move the Mats in an out ofunprocessedImages. - All of the above while wrapping
Matinshared_ptrorunique_ptr(e.g.vector<unique_ptr<Mat>> unprocessedImages;).
None of these attempts had any affect on the characteristics of the memory leak.
Question
What is causing the memory leak in my program? How are you meant to pass ownership of OpenCV Mats between different threads?
Thanks, James
Edit: Added an additional attempted fix.
Edit 2: Running with Valgrind results in the above code NOT leaking. This is backed up by Valgrind's report which states that all blocks were reachable when the program ended. From the programs print outs it is clear that running it in valgrind made it a single threaded application as the print statements inside the two threads are perfectly interleaved.
Edit 3: I modified main like below. The result was that the peak and minimum memory usage on every iteration of the outer loop was the same. On this particular run the minimum memory usage was 374MB.
int main() {
signal(SIGINT, intHandler);
while(keepRunning)
{
runThread = 1;
// Start thread to process images
auto imageProcessingThread = std::thread(image_processing_thread);
/* ... */
cout << "All images processed" << endl;
}
while (keepRunning) {}
return 1;
}