The Computer Language
24.11 Benchmarks Game

Fastest contributed programs, grouped by programming language implementation

Benchmark program elapsed seconds compared to the fastest for selected programming language implementations.

As the median line for Rust fastest contributed-program times is outside the IQR box for C#, there's likely to be a difference between those two groups of measurements.

There's likely to be a difference between the Dart fastest contributed-programs and the Racket and PHP groups :

Benchmark program elapsed seconds compared to the fastest for more programming language implementations.

: and there's a big difference between measurements with the 2008 Matz’s Ruby Interpreter and those with YJIT.


The Geometric Mean can be used to reduce the presentation to one number for each programming language implementation :

Geometric mean of benchmark program elapsed seconds compared to the fastest for selected programming language implementations.

Geometric mean of benchmark program elapsed seconds compared to the fastest for selected programming language implementations.

Some-of the contributed programs were not re-written for multiprocessing, and may seem better if CPU seconds is the performance metric :

Benchmark program CPU seconds compared to the fastest for selected programming language implementations.

Benchmark program CPU seconds compared to the fastest for more programming language implementations.

After all, facts are facts, and although we may quote one to another with a chuckle the words of the Wise Statesman, 'Lies—damned lies—and statistics,' still there are some easy figures the simplest must understand, and the astutest cannot wriggle out of.

Leonard Henry Courtney, 1895

Always look at the source code

Histogram of program elapsed time / fastest elapsed time.

We don't know how much work was done to optimize each program.
 

To fix this, regex crate must be updated or replaced. I spent two weekends on this.

Look at the source code and see which of the faster programs use hand written vector instructions. Look and you might see simpler programs written in the same programming language.
 

… a pretty solid study on the boredom of performance-oriented software engineers grouped by programming language.