undefined | Better HN

0 pointsdmarchand902y ago0 comments

I believe that, contrary to popular belief, the implementation of this system would lead to a substantial increase in productivity in the long run. Here's why:

Currently, a significant proportion of research results in various fields cannot be reproduced. This essentially means that a lot of work turns out to be flawed, leading to wasted efforts (you can refer to the 'reproducibility crisis' for more context). Moreover, future research often builds upon this erroneous information, wasting even more resources. As a result, academic journals get cluttered with substandard work, making them increasingly difficult to monitor and comprehend. Additionally, the overall quality of written communication deteriorates as emphasis shifts from the accurate transfer and reproduction of knowledge to the inflated portrayal of novelty.

Now consider a scenario where 50% of all research is dedicated to reproduction. Although this may seem to decelerate progress in the short term, it ensures a more consistent and reliable advancement in the long term. The quality of writing would likely improve to facilitate replication. Furthermore, research methodology would be disseminated more quickly, enhancing overall research effectiveness.

0 comments

4 comments · 2 top-level

matthewdgreen2y ago· 2 in thread

In the current system scientists allocate reproduction efforts to results that they intend to build on. So if you’ve claimed a breakthrough technique for levitating widgets — and I think this widget technique can be used to build spacecraft (or if I think your technique is wrong) — then I will allocate precious time and resources to reproducing your work. By contrast if I don’t think your work is significant and worth following up on, then I allocate my efforts somewhere else. The advantage is that more apparently-significant results (“might cure cancer”) tend to get a bigger slice of very limited resources, while dead-end or useless results (“might slightly reduce flatulence in cats”) don’t. This distributed entrepreneurial approach isn’t perfect, but it works better than central planning. By contrast you could adopt a Soviet-like approach where cat farts and cancer both share replication resources, but this seems like it would be bad for everyone (except the cats.)

SkyBelow2y ago

>In the current system scientists allocate reproduction efforts to results that they intend to build on. So if you’ve claimed a breakthrough technique for levitating widgets — and I think this widget technique can be used to build spacecraft (or if I think your technique is wrong) — then I will allocate precious time and resources to reproducing your work.

Wouldn't this imply that worthwile results are already being replicated, so the primary cost has been paid and we just need to give some method to disseminate this work and then have it factor into the credibility of science? pre-print < peer reviewed < peer replicated, with the last step having internal rankings depending upon how much it has been replicated?

And it'll also show when someone is building on things but not replicating it, which I guess is an issue in some fields more than others.

GravelRocks2y ago

"Cat fart" research might also be incredibly expensive to replicate compared to the "might cure cancer" research. In that case it would effectively get a bigger slice of resources because we're treating all research the same!

rkangel2y ago

It is analogous to spending half your time writing tests when developing software. Yes it slows you down in the very short term, but in the medium and long term it speeds you up as you have confidence in the code and there aren't bugs lurking everywhere.

j / k navigate · click thread line to collapse