undefined | Better HN

0 pointsTeaBrain1y ago0 comments

>Therefore, (think about the birthday paradox, doesn't matter how many people in the world, a few dozen in the room with you is an adequate sample), it should not surprise you that statistical significance is achieved through a much smaller sample size than most non-statisticians have intuition for.

This response on the supposed the lack of importance of a sample size is completely wrong on just about every claim. The parent comment had a valid point. Just because a population may fit a certain distribution, does not mean that any given sample size will also fit that distribution. Samples are used to ideally create a representative group of a population, that is smaller than the population. However, the sample size required to come close to a representative distribution can vary between populations and variables being examined. Also, using the birthday paradox is a terrible example and has nothing to do with statistical significance, as the so-called birthday paradox is just a simple function.

0 comments

3 comments · 1 top-level

braiamp1y ago· 2 in thread

Except that sample size doesn't matter if the set of potential results/measurements of the dependent variable are very large. Someone pointed out that you can demonstrate that alcohol impairs executive functions, balance, etc. with a very small sample size, because the effects would be so large and evident that your statistical power would be. On very large variance, where the results are dichotomous in nature (can a subject walk straight in a 10 meters line, without walking outside: yes/no) can have a very small sample size.

Use this calculator, set options to: two independent groups, dichotomous, group 1 = 90%, group 2 = 10%, incidence, enrollment ratio = 1, alpha = 0.05 and power = 80%. The sample size is 10, 5 for each group. https://clincalc.com/stats/samplesize.aspx

TeaBrainOP1y ago

That other comment on alcohol is by the same guy that made the comment that I responded to here. The same issue there is that result reliability can be influenced by the effect size of the variables being tested, but this still is far from a guarantee that the results will generalize, which is more likely to be an issue with a smaller sample.

An issue with the comment I previously responded to, as I mentioned above, is that they made it out as if a small sample size could be reliable to determine a reliable statistical significance irrespective of the variables under study and tried to prove this by using an absurd analogy between statistical significance and the birthday paradox. The problem with their attempted point was that it didn't even respond to the comment above it, which pointed out that a high statistical significance is not a guarantee of reproducibility, especially with a low sample size.

braiamp1y ago

Sample size only matters for statistical power. After certain point, the diminishing returns will fall of a cliff. Lay persons seems to have mistrust of seemly small sample sizes, but they need to understand that doubling the size of the study only improves statistical in 5-10% for dichotomous studies. If we mandated a bigger sample size without taking into account why it's relevant and how it's calculated, it will make every study cost prohibited.