What is performance penalty for converting a function into coroutine in C++?

Viewed 1138

Since for a C++20 coroutine the compiler has to create a different code than for an ordinary function by putting all local variables in heap instead of stack, what is expected slowdown of coroutine function relative to an ordinary function making the same computation?

I wrote a simple test that measures the time of summation of a large number of values:

#include <iostream>
#include <chrono>
#include <coroutine>

constexpr size_t N = 1024ull*1024ull*1024ull;

double compute()
{
    double res = 0;
    for ( size_t i = 0; i < N; ++i )
        res += i;
    return res;
}

template<typename F>
auto timer( F f )
{
    using namespace std::chrono;
    auto s = high_resolution_clock::now();
    auto res = f();
    nanoseconds dur = high_resolution_clock::now() - s;
    std::cout << "duration: " << dur.count() * 1e-9 << " sec\n";
    return res;
}

struct Future {
  struct promise_type {
    double value_;
    Future get_return_object() { return { std::coroutine_handle<promise_type>::from_promise(*this) }; }
    std::suspend_always initial_suspend() { return {}; }
    std::suspend_always final_suspend() noexcept { return {}; }
    void unhandled_exception() {}
    void return_value(double v) { value_ = v; }
  };
  bool done() { return h_.done(); }
  void resume() { return h_(); }
  double value() { return h_.promise().value_; }
  ~Future() { h_.destroy(); }
  std::coroutine_handle<promise_type> h_;
};

double computeInCoroutine()
{
    auto future = []() -> Future
    {
        double res = 0;
        for ( size_t i = 0; i < N; ++i )
            res += i;
        co_return res;
    }();
    while (!future.done())
        future.resume();
    return future.value();
}

int main()
{
    std::cout << "Ordinary function ";
    auto res = timer( compute );
    std::cout << "Coroutine ";
    res += timer( computeInCoroutine );
    return (int)res;
}

In Visual Studio 2019 16.10.3 I get the result:

Ordinary function duration: 0.793619 sec
Coroutine duration: 1.05897 sec

Please note that the times were obtained on a local computer and they are very repeatable. I would not recommend measure the time online (e.g. in godbold.org), since it is very unstable there.

So just by converting an ordinary function in a coroutine that does not suspend in the middle of computation, in MSVC we get about 30% of performance penalty, or the comparison is not fair?

Update. After I increased the number of iterations in 16 times:

constexpr size_t N = 16ull*1024ull*1024ull*1024ull;

the difference in performance of two functions became really dramatic:

Ordinary function duration: 12.602 sec
Coroutine duration: 45.5615 sec
2 Answers

I took your code and made it work in clang/gcc on godbolt.

Clang:

Ordinary function duration: 2.27166 sec
5.76461e+17
Coroutine duration: 2.76769 sec
5.76461e+17

gcc:

Ordinary function duration: 2.21894 sec
5.76461e+17
Coroutine duration: 2.18039 sec
5.76461e+17

Clang was 23% slower, gcc was about the same speed (the difference is noise).

When I enabled -ffast-math, I get:

Ordinary function duration: 0.465791 sec
5.76461e+17
Coroutine duration: 1.58706 sec
5.76461e+17

in clang, and

Ordinary function duration: 2.19963 sec
5.76461e+17
Coroutine duration: 2.23827 sec
5.76461e+17

in gcc.

I then rewrote the future to read:

auto future = []() -> Future
{
    auto helper = [](){
        double res = 0;
        
        for ( size_t i = 0; i < N; ++i )
            res += i;
        return res;
    };
    co_return helper();
}();

where I hide the state from the coroutine in a helper lambda. This changes the clang timings to:

Ordinary function duration: 1.07504 sec
5.76461e+17
Coroutine duration: 0.465179 sec
5.76461e+17

The assembly makes nothing obvious. So I swapped their order (without hiding the lambda) and got:

Coroutine duration: 0.467913 sec
5.76461e+17
Ordinary function duration: 0.959462 sec
5.76461e+17

and running them in the opposite order makes the first one faster.

Thus, we are seeing artifacts of the performance test harness.

You need to test this cross-compiler, as coroutine support is new and improving. You need to use a real test harness, and multiple runs, and complete symmetry between the two cases (not run one, then the other; only run one on any one execution of the program). And you need to set optimization flags, and tweak the details of your micro optimization to see if there is an instability.

Then you need to look at the generated assembly to understand if the cause of the slowdown is plausible, and where it might be.

Micro optimization is hard.

In any case I managed to get speeds matching or exceeding the non-coroutine version as I tweaked the benchmark in a bunch of ways. So no, a 30% hit for such a case is not to be expected.

Writing this as a proper benchmark (with clang compatibility changes thanks to @Yakk's answer):

#include <benchmark/benchmark.h>
#include <iostream>
#include <chrono>

#ifdef __clang__
#include <experimental/coroutine>
#else
#include <coroutine>
#endif

namespace cor {
    #ifdef __clang__
    using namespace std::experimental;
    #else
    using namespace std;
    #endif
}

constexpr size_t N = 1024ull*1024ull*1024ull;

struct Future {
  struct promise_type {
    double value_;
    Future get_return_object() { return { cor::coroutine_handle<promise_type>::from_promise(*this) }; }
    cor::suspend_always initial_suspend() { return {}; }
    cor::suspend_always final_suspend() noexcept { return {}; }
    void unhandled_exception() {}
    void return_value(double v) { value_ = v; }
  };
  bool done() { return h_.done(); }
  void resume() { return h_(); }
  double value() { return h_.promise().value_; }
  ~Future() { h_.destroy(); }
  cor::coroutine_handle<promise_type> h_;
};

double compute()
{
    double res = 0;
    for ( size_t i = 0; i < N; ++i )
        benchmark::DoNotOptimize(res += i);
    return res;
}

double computeInCoroutine()
{
    auto future = []() -> Future
    {
        double res = 0;
        for ( size_t i = 0; i < N; ++i )
            benchmark::DoNotOptimize(res += i);
        co_return res;
    }();
    while (!future.done())
        future.resume();
    return future.value();
}

static void BenchCompute(benchmark::State& state) {
  for (auto _ : state) {
    benchmark::DoNotOptimize(compute());
  }
}
BENCHMARK(BenchCompute);

static void BenchComputeCoroutine(benchmark::State& state) {
  for (auto _ : state) {
    benchmark::DoNotOptimize(computeInCoroutine());
  }
}

BENCHMARK(BenchComputeCoroutine);
BENCHMARK_MAIN();

I obtain the following results with MSVC 19 x64 Release:

2021-07-27T13:22:02-04:00
Running <some_path>\bench.exe
Run on (24 X 3793 MHz CPU s)
CPU Caches:
  L1 Data 32 KiB (x12)
  L1 Instruction 32 KiB (x12)
  L2 Unified 512 KiB (x12)
  L3 Unified 16384 KiB (x4)
----------------------------------------------------------------
Benchmark                      Time             CPU   Iterations
----------------------------------------------------------------
BenchCompute          2698332000 ns   2687500000 ns            1
BenchComputeCoroutine 2700581500 ns   2703125000 ns            1

GCC and clang results would help complete the picture, but it seems clear to me that your test is done in a way that overstates the overhead of the coroutine, at least on MSVC.

Edit: If I remove the "busy work" from the compute() functions and benchmark nothing but the overhead.

double computeInCoroutine()
{
    auto future = []() -> Future
    {
        double res = 0;
        co_return res;
    }();
    while (!future.done())
        future.resume();
    return future.value();
}

double compute()
{
    double res = 0;
    return res;
}

I obtain the following:

2021-07-27T13:47:29-04:00
Running <some_path>\bench.exe
Run on (24 X 3793 MHz CPU s)
CPU Caches:
  L1 Data 32 KiB (x12)
  L1 Instruction 32 KiB (x12)
  L2 Unified 512 KiB (x12)
  L3 Unified 16384 KiB (x4)
----------------------------------------------------------------
Benchmark                      Time             CPU   Iterations
----------------------------------------------------------------
BenchCompute                2.61 ns         2.61 ns    263529412
BenchComputeCoroutine       66.5 ns         66.3 ns      8960000

Which indicates a 60-ish nanosecond overhead (compiler and system-specific). Still much lighter than what OP is reporting.

Related