I'm writing a path tracer as a programming exercise. Yesterday I finally decided to implement multithreading - and it worked well. However, once I wrapped the test code I wrote inside main() in a separate renderer class, I noticed a significant and consistent performance drop. In short - it would seem that filling std::vector anywhere outside of main() causes threads using its elements to perform worse. I managed to isolate and reproduce the issue with simplified code, but unfortunately I still don't know why it happens or what to do in order to fix it.
Performance drop is quite visible and consistent:
97 samples - time = 28.154226s, per sample = 0.290250s, per sample/th = 1.741498
99 samples - time = 28.360723s, per sample = 0.286472s, per sample/th = 1.718832
100 samples - time = 29.335468s, per sample = 0.293355s, per sample/th = 1.760128
vs.
98 samples - time = 30.197734s, per sample = 0.308140s, per sample/th = 1.848841
99 samples - time = 30.534240s, per sample = 0.308427s, per sample/th = 1.850560
100 samples - time = 30.786519s, per sample = 0.307865s, per sample/th = 1.847191
The code I originally posted in this question can be found here: https://github.com/Jacajack/rt/tree/mt_debug or in edit history.
I created a struct foo that is supposed to mimic the behavior of my renderer class and is responsible for initialization of path tracing contexts in its constructor.
The interesting thing is, when I remove the body of foo's constructor and instead do this (initialize contexts directly from main()):
std::vector<rt::path_tracer> contexts; // Can be on stack or on heap, doesn't matter
foo F(cam, scene, bvh, width, height, render_threads, contexts); // no longer fills `contexts`
contexts.reserve(render_threads);
for (int i = 0; i < render_threads; i++)
contexts.emplace_back(cam, scene, bvh, width, height, 1000 + i);
F.run(render_threads);
the performance is back to normal. But then, if I wrap these three lines into a separate function and call it from here, it's worse again. The only pattern I can see here is
that filling the contexts vector outside of main() causes the problem.
I initially thought that this was an alignment/caching issue, so I tried aligning path_tracers with Boost's aligned_allocator and TBB's cache_aligned_allocator with no result. It turns out that this problem persists even when there's only one thread running.
I suspect it must be some kind of wild compiler optimization (I'm using -O3), althought that's just a guess. Do you know any possible causes of such behavior and what can be done to avoid it?
This happens on both gcc 10.1.0 and clang 10.0.0. Currently I'm only using -O3.
I managed to reproduce a similar issue in this standalone example:
#include <iostream>
#include <thread>
#include <random>
#include <algorithm>
#include <chrono>
#include <iomanip>
struct foo
{
std::mt19937 rng;
std::uniform_real_distribution<float> dist;
std::vector<float> buf;
int cnt = 0;
foo(int seed, int n) :
rng(seed),
dist(0, 1),
buf(n, 0)
{
}
void do_stuff()
{
// Do whatever
for (auto &f : buf)
f = (f + 1) * dist(rng);
cnt++;
}
};
int main()
{
int N = 50000000;
int thread_count = 6;
struct bar
{
std::vector<std::thread> threads;
std::vector<foo> &foos;
bool active = true;
bar(std::vector<foo> &f, int thread_count, int n) :
foos(f)
{
/*
foos.reserve(thread_count);
for (int i = 0; i < thread_count; i++)
foos.emplace_back(1000 + i, n);
//*/
}
void run(int thread_count)
{
auto task = [this](foo &f)
{
while (this->active)
f.do_stuff();
};
threads.reserve(thread_count);
for (int i = 0; i < thread_count; i++)
threads.emplace_back(task, std::ref(foos[i]));
}
};
std::vector<foo> foos;
bar B(foos, thread_count, N);
///*
foos.reserve(thread_count);
for (int i = 0; i < thread_count; i++)
foos.emplace_back(1000 + i, N);
//*/
B.run(thread_count);
std::vector<float> buffer(N, 0);
int samples = 0, last_samples = 0;
// Start time
auto t_start = std::chrono::high_resolution_clock::now();
while (1)
{
last_samples = samples;
samples = 0;
for (auto &f : foos)
{
std::transform(
f.buf.cbegin(), f.buf.cend(),
buffer.begin(),
buffer.begin(),
std::plus<float>()
);
samples += f.cnt;
}
if (samples != last_samples)
{
auto t_now = std::chrono::high_resolution_clock::now();
std::chrono::duration<double> t_total = t_now - t_start;
std::cerr << std::setw(4) << samples << " samples - time = " << std::setw(8) << std::fixed << t_total.count()
<< "s, per sample = " << std::setw(8) << std::fixed << t_total.count() / samples
<< "s, per sample/th = " << std::setw(8) << std::fixed << t_total.count() / samples * thread_count << std::endl;
}
}
}
and results:
For N = 100000000, thread_count = 6
In main():
196 samples - time = 26.789526s, per sample = 0.136681s, per sample/th = 0.820088
197 samples - time = 27.045646s, per sample = 0.137288s, per sample/th = 0.823725
200 samples - time = 27.312159s, per sample = 0.136561s, per sample/th = 0.819365
vs.
In foo::foo():
193 samples - time = 22.690566s, per sample = 0.117568s, per sample/th = 0.705406
196 samples - time = 22.972403s, per sample = 0.117206s, per sample/th = 0.703237
198 samples - time = 23.257542s, per sample = 0.117462s, per sample/th = 0.704774
200 samples - time = 23.540432s, per sample = 0.117702s, per sample/th = 0.706213
It seems that the results are the opposite of what is happening in my path tracer, but the visible difference is still here.
Thank you