The Computer Language
25.02 Benchmarks Game

Measured : Which programming language is fastest?


… and the computer language benchmarks game are examples of micro benchmarks. These are easy to use, easy to measure, but far from realistic. They are nonetheless valuable tools.

My question is if anyone here has any experience with simplistic benchmarking and could tell me which things to test for in order to get a simple idea of each language's general performance?

Time to Solve TTS. Is that lowest Elapsed Time or lowest CPU Time or mean CPU Time? Optimized, Un-optimized or Goldilocks programs.

There's more than one right answer.

Answer for a few un-optimised fannkuch-redux programs —


Un-optimised single-thread #8 programs transliterated line-by-line literal style into different programming languages from the same original.

  source code secs mem gz    
  C gcc #8 39.34 2,343 483    
  Java #8 46.66 59,994 517    
  Node.js #8 58.20 63,136 452    
  PHP #8 14 min 16,364 458    
  Ruby yjit #8 34 min 17,691 418    
  Python 3 #8 40 min 11,874 414    

Or choose your answer for Rust and Julia and Haskell and … crowd sourced fannkuch-redux programs.

Answer for the fastest contributed programs —


Seven simple tasks. A couple of dozen different programming-language implementations. For each language implementation, the normalized performance of the fastest contributed program for each task.

Explore the measurements with box plot charts — medians, dispersion, skew.


Benchmark program elapsed seconds compared to the fastest for more programming language implementations.

Programs for binary-trees, pidits and regex-redux have not been included, because the differences in libraries and approach would be hidden in the box plot summary.

The median-line for some language implementation groups seems to fall outside the IQR box for other groups. Choose your answer for likely differences between the groups.

Answer for A versus B —


Side-by-side comparison tables for the most popular searches.

Differences in approach — to memory management, parallel programming, regex, arbitrary precision arithmetic, implementation technique — are part and parcel of using different programming languages.

Choose your answer for the particular comparison that interests you.


And, also, as-well —

Tasks

in-significant io —

significant-io —

contentious —

More Measurements & Comparisons —

Concern has been expressed that short labels (like Java) demonstrate a confusion between programming language and language implementation. These short labels are just short. Each refers to a specific version of a programming language implementation.