How much difference does it make for these tiny programs? otoh For measurements of a few tenths of a second, a few tenths of a second is a huge difference. otoh For measurements of seconds and tens-of-seconds, JVM startup, JIT, OSR… are quickly effective; and tiny differencies amortize.
N=12 fannkuch-redux #8 jdk-23, the 1st and 5th in-process measurements were the only ones below 54 seconds:
| mean
| 55.336
|
| 1
| 52.561
|
| 2
| 54.186
|
| 3
| 54.926
|
| 4
| 50.895
|
| 5
| 55.906
|
| 6
| 55.003
|
| 7
| 56.081
|
| 8
| 54.919
|
| 9
| 55.805
|
| 10
| 54.979
|
Some experimental studies show 10.9% of process executions don't reach a steady state of peak performance; and 43.5% of process executions were inconsistent; and sometimes they are slower than what came before
.
Let's compare the fastest no warmup measurements against the fastest JMH after startup & warmup SampleTime p(0.0000) and mean measurements:
| java 22 2024-03-19
| SampleTime
| Mean
|
| i5-3330
| No warmup
| JMH after startup & warmup
|
| fannkuch-redux #1
| 10.145
| 9.815
| 10.343 N = 35
|
| fannkuch-redux #2
| 46.275
| 42.480
| 43.100 N = 25
|
| fannkuch-redux #3
| 40.162
| 39.796
| 40.378 N = 25
|
| n-body #1
| 7.892
| 7.718
| 7.740 N = 50
|
| n-body #2
| 7.577
| 7.332
| 7.361 N = 50
|
| n-body #3
| 7.624
| 7.374
| 7.390 N = 50
|
| n-body #4
| 6.905
| 6.795
| 6.827 N = 50
|
| n-body #5
| 6.793
| 6.677
| 6.689 N = 50
|
| spectral-norm #1
| 7.074
| 6.795
| 6.800 N = 50
|
| spectral-norm #2
| 2.375
| 1.950
| 2.110 N = 131
|
| spectral-norm #3
| 1.681
| 1.569
| 1.593 N = 175
|
