Lately, I started my pleasant experience with Github actions to automate tests execution.
Thus, I took a step forward and I successfully made some experiences running benchmarks, namely with JMH. From some samples I can observe consistent benchmark results between my Windows desktop running Java 11 and those ones gathered with Github actions running Ubuntu with Java 11 too.
Yet, I am not entirely confident about these benchmarks results from Github Actions!
How reliable should I consider my observations on Github Actions running benchmarks?
Completing my question, I would like to point that there is a big advantage on having such a platform to run benchmarks with publicly available output. Even when developers turn their benchmarks suites available to the community there is always a suspicious about each running environment and why each developer claims to have different results from the other, for the same benchmark and configurations. Using such platform as Github Actions, we are not only sharing our running environment, but most important we are also providing the logging output which shows unbiased results, suppressing eventual suspicious