I think it's fascinating that we can at the same time hold things like "one is none" to be true, or that you should write tests first, but with science we already got so used to a lack of discipline that we just declare it fine.
It's not hard to not climb a tower you can't get down from. It's the default, actually. You start with something small where you can describe everything that goes into replicating it. Then you replicate it yourself, based on your own instructions. Before that, you don't bother anyone else with it. Once that is done, and others can replicate as well, it "actually exists".
And if that means the majority of stuff has to be thrown out, I'd suggest doing that sooner rather than later, instead of just accumulating scientific debt.
That is not to say that scientist should not document the process to their best ability so it can be reproduced in principle. I'm just arguing that it is impossible to easily reproduce other people's results. Again when chemical/manufacturing companies open another location they often spend months to years to make the process work in the new factory.
We're not talking about 1 of 10 reproduction attempts failing, we're talking about 100%. And no, companies don't time and time again try to reproduce something that has never been reproduced and fail, to then try again, endlessly. That's just not a thing.
> it is impossible to easily reproduce other people's results
We're also not talking about "easily" reproducing something, but at all. And in principle doesn't cut it, it needs to be reproduced in practice.
Meanwhile, Alice has spent the last 5 years making the best possible use of the flawed theory, and published a lot of original research. Sure, many of her publications are rubbish, but a few contain interesting results. Contrary to Bob, Alice can show actual results and has publications.
Who do you believe will remain in academia? And, according to public perception, will seem more like an actual scientist?
Academic science isn’t just the doing science part but the articulation and presentation of your work to the broader community. If Bob knows this space so well, he should be able to clearly communicate the issue and, ideally, present an easily understandable counter example to the existing theory.
Technical folks undervalue presentation when writing articles and presenting at conferences. The burden of proof is on the presenter, and, unless there’s some incredible demonstration at the end, most researchers won’t have the time or attention to slog through your mess of a paper to decipher it. There’s only so much time in the day and too many papers to read.
In my experience, the best researchers are also the best presenters. I’ve been to great talks out of my domain that I left feeling like I understood the importance of their work despite not understanding the details. I’ve also seen many talks in my field that I thought were awful because the presentation was convoluted or they didn’t motivate the importance of their problem / why their work addressed it
To put a point on it, if public distrust of science becomes big enough, it all can go away before you can say "cultural revolution" or "fascist strongman". Then there'd be no more academia, and its shell would be inhabited by party members, so to speak. I'd gladly sacrifice the ability of Alice and others like her to live off producing "mostly rubbish" to at least have a chance to save science itself.
