Why are Bench module's "wallclock" seconds report so wildly off?

user_1641112 0
Viewed 103

I'm benchmarking this code:

use Bench;
my $b = Bench.new;
say 'start';
my $i=0;
$b.timethese(100000, {
    first => sub { while $i++ < 10000 { }   },
    second => sub { while $i++ < 10000 { }  }
});

I get this report:

Timing 100000 iterations of first, second...
     first: 2.105 wallclock secs (1.786 usr 0.449 sys 2.235 cpu) @ 47502.644/s (n=100000)
    second: 2.341 wallclock secs (2.012 usr 0.445 sys 2.458 cpu) @ 42710.876/s (n=100000)

So about 4.5 "wallclock" seconds total. But by my stopwatch, it's is taking around 19 seconds. What accounts for this?

raku
1 Answers

Aiui:

  • The Bench module uses¹ the Telemetry module.

  • While Telemetry provides very precise measurement of performance, it introduces overhead in doing so, and that overhead can be very large compared to the time/memory consumed by the code being measured.

  • The "wallclock" seconds Bench is reporting are what the "wallclock" seconds would be if the Telemetry overhead were not incurred.

¹ Or, more accurately, trys to require it.

user_10776720
Related