undefined | Better HN

0 pointsvidarh12y ago0 comments

GeneralMayhem pointed out that this is flat out wrong. Here's an example to demonstrate it:

Consider a population where 99% of users use IE, and 1% of users use Chrome.

From that population you can draw a biased sample consisting of the 99% IE users and conclude that there are no Chrome users at all.

Or you could draw a random sample of 1% of the population, and assuming that 1% adds up to enough people, your chances of drawing a random sample that did not include a reasonable number of Chrome users too would be extremely low.

So depending on methods used, even a 99% sample may be entirely meaningless when compared to a 1% sample.

0 comments

1 comments · 1 top-level

FootballMuse12y ago

If you extrapolate the raw numbers, then yes, you can conclude there are no Chrome users at all. But no one would accept this as fact knowing the sampling methodology.

You can, however, still build confidence intervals based on biased samples. In your example, with 100% certainty, between 0-1% of the population uses Chrome. Yet of course, even the 99.999% CI will be wrong due to the severe bias. Now, if in your example only a 1% biased sample were looked at, the 100% CI would be 0-99%. Much less information. Note that you may also still see trends in biased samples if the sample is consistent.

If biased samples were meaningless, then how are Stat Counter or Net Applications results valuable at all since they are not random samples?

j / k navigate · click thread line to collapse