undefined | Better HN

0 pointsmarcosdumay5y ago0 comments

Due to the sheer number of hypotheses get tested all the time in particle physics, 3 sigma effects appear all the time, and then disappear before getting a larger certainty. That's why it's considered a suggestion, not proof.

And yes, the exact numbers are arbitrary, but their ballpark isn't. You reduce the necessary certainty if you have confidence on your priors, and increase it if you test many different hypotheses. The target confidence also varies from one discipline to another based on how much data one can realistically gather, but nearly all of physics falls on the "we can gather enough data" category anyway.

0 comments

4 comments · 3 top-level

ramraj075y ago· 1 in thread

Well put, but I’ve always felt a middle ground existed where a knowledgeable scientist can make a very educated guess with much lesser statistical significance as long as the experiment is thought through, you are aware of the noise models and alternative explanations have been explored and mostly ruled out. Of course for publication you better whip your ass to whatever sigma they demand, but more often than not there either cause for optimism or pessimism ways before that point.

marcosdumayOP5y ago

Not really. The amount of confidence you have grows exponentially with the observed deviation, and that last one grows kinda proportionally to the amount of data you have (if the hypothesis is real, of course).

So there is some fine adjustment possible, but it's very small on the experiments where you need a lot of confidence. There is more margin to judge low non-significant results at the low confidence end, as exponentials grow slowly when they are small.

panda-giddiness5y ago

> Due to the sheer number of hypotheses get tested all the time in particle physics, 3 sigma effects appear all the time, and then disappear before getting a larger certainty.

See, eg, the look-elsewhere effect [1] or the problem of multiple comparisons [2].

---

[1] https://en.wikipedia.org/wiki/Look-elsewhere_effect

[2] https://en.wikipedia.org/wiki/Multiple_comparisons_problem

a9h74j5y ago

Re: arbitrary. I once attended a Stephen Toulman lecture where he explored the meaning of 'arbitrary.' While the word is often taken to connote "random and uninformed", the root of the word indeed connotes "arbitration," i.e. informed judgement.

j / k navigate · click thread line to collapse