undefined | Better HN

0 pointstlarkworthy12y ago0 comments

"average and std dev are only revelent if the distribution is Gaussian in distrition"

technically not true. Knowledge of the second order moment (variance) lets you uniquely identify other distributions like Poisson, or uniform. Knowledge of even higher order moments lets you fit more complicated statistical models.

Low variance is good, regardless of underlying distribution.

0 comments

mynegation12y ago

Probably the statement should be that comparing mean and variance are only relevant if both metrics follow the same distribution. In the absence of distribution information (and it is usually absent in empirical tests like that) quantiles would help to do a better job at comparing performance.

diroussel12y ago

Indeed. Often latency measurements are clumpy.

You might have one clump of fast responses when no GC occurs, another when some GC occurs, and a smaller clump where a stop-the-world full GC has occurred.

In such a case average is not meaningful.

j / k navigate · click thread line to collapse