undefined | Better HN

0 pointsa_bonobo12y ago0 comments

There's a follow-up paper to the one you linked that claims that much less published research findings are false:

>We estimate that the overall rate of false discoveries among reported results is 14% (s.d. 1%), contrary to previous claims. We also found that there is no a significant increase in the estimated rate of reported false discovery results over time (0.5% more false positives (FP) per year, P=0.18) or with respect to journal submissions (0.5% more FP per 100 submissions, P=0.12).

[1] http://biostatistics.oxfordjournals.org/content/early/2013/0...

The circle has begun :)

0 comments

capnrefsmmat12y ago

You should also read the responses to that article, such as Ioannidis's, which is scathing. They're all open-access, thankfully, and you can find them here:

http://simplystatistics.org/2013/09/25/is-most-science-false...

Jager and Leek put their code on GitHub, so the commenters were able to review their code and tinker with it to see what would happen.

a_bonoboOP12y ago

Thanks for all these links! I skimmed them, and it's fun to see that two of them rip into Ionnadis' work, while Ionnadis rips into the Jager/Leek work... It's also a cool exercise in reproducible research and open peer review, both of which are far from common.

In my opinion, the entire exercise of data-mining the published literature is pretty much futile. We already know there are problems in the published literature and that scientists are pretty mediocre at statistics. Pin-pointing the exact value of how mediocre only leads to, as your link shows, a mountain of published works, hurt feelings (on Ionnadis side I guess) and doesn't solve anything.

j / k navigate · click thread line to collapse