A month ago: https://news.ycombinator.com/item?id=31828509
Potential fabrication in research threatens the amyloid theory of Alzheimer’s - https://news.ycombinator.com/item?id=32183302 - July 2022 (236 comments)
Alzheimer’s amyloid hypothesis ‘cabal’ thwarted progress toward a cure (2019) - https://news.ycombinator.com/item?id=31828509 - June 2022 (307 comments)
How an Alzheimer’s ‘cabal’ thwarted progress toward a cure - https://news.ycombinator.com/item?id=21911225 - Dec 2019 (382 comments)
The amyloid hypothesis on trial - https://news.ycombinator.com/item?id=17618027 - July 2018 (43 comments)
Is the Alzheimer's “Amyloid Hypothesis” Wrong? (2017) - https://news.ycombinator.com/item?id=17444214 - July 2018 (109 comments)
These comments are written by real Alzheimer's researchers. They all disagree with the notion that Lesné's papers have been important to the field, and therefore undermine the idea that this has any bearing on "two decades of Alzheimer's research". (Karen Ashe, co-author of the main paper referenced here, also stops by the thread.)
[1] https://www.alzforum.org/news/community-news/sylvain-lesne-w...
We can blame regulatory capture of the FDA for approval of failed drugs, rather than the scientific establishment. And we can blame Sylvain Lesné for Sylvain Lesné's fraud.
I would encourage those of us whose only knowledge of this topic is the word "amyloid" (I admit I am one of them) to read the scientists' comments and appreciate that there is more to this than we know. There are complexities, nuances, diverse perspectives and healthy disagreements. It's not just a political battlefield. Projecting culture war into it would be harmful to the scientific progress we all value and to the millions who suffer from Alzheimer's.
They have an interest in limiting the scope of the perception of fraud in the field.
Alzheimer’s amyloid hypothesis ‘cabal’ thwarted progress toward a cure (2019) https://news.ycombinator.com/item?id=31828509
How an Alzheimer’s ‘cabal’ thwarted progress toward a cure https://news.ycombinator.com/item?id=21911225
The amyloid hypothesis on trial https://news.ycombinator.com/item?id=17618027
Is the Alzheimer's “Amyloid Hypothesis” Wrong? (2017) https://news.ycombinator.com/item?id=17444214
Why, at this point should we believe any one scientist writing in that forum wasn't already sunk far deep into amyloid research in their career?
Not to mention that after a quick glance on the comments section, I fail to see where you get the idea that "They all disagree with the notion that Lesné's papers have been important to the field". Apart from the very first comment from karen Ashe (who will obviously be defending her research) and a few other who's working on related topics, other commenters seems to be keeping their suspicion at amyloid hypothesis.
It's also plenty obvious that there is no single, monolithic "current research direction" or even that this researcher's work was of fundamental impact when it was published - not to mention the number of people that were highly skeptical from the beginning.
"Alzheimer’s amyloid hypothesis ‘cabal’ thwarted progress toward a cure (2019)"
https://news.ycombinator.com/item?id=31828509
https://news.ycombinator.com/item?id=21911225
(also see dang's comment who lists 2 more thread https://news.ycombinator.com/item?id=32213973)
Parent comment is downvoted but it seems like most alzheimer's researchers have vested interest in amyloid hypothesis one way or the other.
[1] https://www.dailykos.com/stories/2022/7/22/2111914/-Two-deca...
[1] https://www.science.org/content/article/potential-fabricatio...
https://www.science.org/content/article/potential-fabricatio...
Academia and research needs a new broom. Presently incentives are peverse. Impact factors, publisher corruption, grant applications and funding are a blight on science.
Also remember that Kickstarter found it necessary to write a blog post “Kickstarter is not a store“, because people were expecting projects to return fast, reliable, tangible results. By its nature, science will be even further from that revealed preference of funders
My opinion: capitalism has corrupted literally every single thing it has touched.
> "The suspicion that something was more than a little wrong with the model that is getting almost all Alzheimer’s research funding ($1.6 billion in the last year alone) began with a fight over the drug Simufilam. The drug was being pushed into trials by its manufacturer, Cassava Sciences, but a group of scientists who reviewed the drug maker’s claims about Simufilam believed that it was exaggerating the potential. So they did what any reasonable person would do: They purchased short sell positions in Cassava Sciences stock, filed a letter with the FDA calling for a review before allowing the drug to go to trial, and hired an investigator to provide some support for this position."
However, the desire to gain a profit by pushing a questionable drug through trials was also involved. I'd note however, that in the Soviet Union, the likes of Lysenko also pushed fraudulent research in order to improve their standing in the Soviet heirarchy, which came with various rewards.
People like wealth and power, and some will do anything to get it, regardless of the nature of the society they live in.
Funding for these projects is often disconnected from the market.
They are usually dispersed by massive bureaucratic agents with incentive structures that have nothing to do with profit (more internal politics and prestige).
Imagine if the human body didn’t have a dozen different organs each producing a specialized product for your body and instead had one organ that tried to do everything?
Blaming “capitalism” is just the “old man yelling at cloud” if you are 20.
https://physicstoday.scitation.org/doi/full/10.1063/1.324848...
A key point, from the above review, that I think explains a lot of this behavior:
> "Reich points out that fraudsters like Schön could get credit for “first discovery” if, before they are caught, their false claims are confirmed by others on the basis of genuine data."
It did shake up the field of organic semiconductor device research in physics by increasing scrutiny and changing some requirements (for example, electron microscope imagery of claimed devices is now a requirement for publication). However, as the top post at present notes, the incentives are backwards in academic science these days, and the role of funding organizations and high-profile journals is as problematic as that of the originating fraudsters.
Maybe this instance of fraud will do the same for the biomedical field, by forcing researchers to release their raw data and full-resolution images as a condition of publication, although that would require a major shift in behavior in today's patent-driven startup-centered heavily-corporatized biomedical research world.
Personally, I'll note that during the years I worked in academia, of the three PIs I worked with, I discovered two engaging in fraudulent research to greater or lesser extent. The main differences between them and the one who wasn't were (1) lab notebook discipline and recording and storing data securely, (2) in-house replication was required, (3) no toleration for BS and shady behavior. The others broke all those rules. (Unfortunately I picked the wrong PI to work with, and ended up leaving academia in a fit of contemptuous disgust.)
A good rule of thumb: If some research claim hasn't been replicated, and if the data and methods aren't transparently available, then it's as likely to be fraudulent garbage as not, and it's not worthy of further examination.
Fraud by false representation
(1) A person is in breach of this section if he—
(a) dishonestly makes a false representation, and
(b) intends, by making the representation—
(i) to make a gain for himself or another
It really lays open how easy it is to mislead everyone. All these siloed scientists won't have a clue anything is wrong. This is how conspiracies would work... if there is advantage to someone somewhere and they have the means to alter the model in their favour, why wouldn't they?
I suspect there was deliberate fraud, but this article doesn't provide any more evidence of that than previous articles.
> Since that 2006 publication, the presence or absence of this specific amyloid has often been treated as diagnostic of Alzheimer’s. Meaning that patients who did die from Alzheimer’s may have been misdiagnosed as having something else. Those whose dementia came from other causes may have falsely been dragged under the Alzheimer’s umbrella.
I think the author is confused about the controversy he is reporting on. Nobody is suggesting that there aren't elevated levels of Aβ in Alzheimer's brains. The controversy is only about the presence of Aβ56, and as far as I know Aβ56 was never used to diagnose Alzheimer's disease. It should also be noted that this is only relevant to postmortem diagnosis, so even if they were testing for Aβ*56 it wouldn't have affected the diagnosis of living patients.
At the bottom of the article is a note, "Article written by Mark Sumner via Daily Kos". This explains a lot. Daily Kos is a site that got its start with sensationalized political articles. Now they've apparently expanded to subjects where they can do more damage.
---
As I post the comment, the title of the linked article is: "Two decades of Alzheimer’s research was based on deliberate fraud by 2 scientists that has cost billions of dollars and millions of lives"
[0] https://www.science.org/content/article/potential-fabricatio... [1] https://news.ycombinator.com/item?id=32183302
People do this all the time. In fact I’d say it’s the usual reaction when confronted with data contradicting one’s beliefs.
Scientists are supposed to learn to go past that but I wonder how many actually do, especially when there is both social and economic pressure to conform to a school of thought.
Looking at reality in an unbiased way and trying to draw rational conclusions is incredibly rare and requires effort. Ego and social group pressure are the enemy.
Cackling evildoers are also very rare. Most evil is a product of how we normally behave and of normal social and economic incentives.
These people need to be formally immediately banned from any NIH activity, and criminally charged. We have known for years that their crap work was useless.
„I sincerely doubt that the absence of this particular paper and AB*56 from historical scientific record would have significantly changed the last 20 years of AD drug development. That is because there is strong genetic and other evidence for the role of amyloid in disease.“
Want to see amyloid defenders? Read comments on Derek Lowe's blog, and there's quite a few commenters there who are arguing that a better amyloid drug that actually clears out the plaques is needed (apparently not realizing that's what Aduhelm, and it still didn't work).
Highly recommend this book to discuss why: https://www.amazon.com/Rigor-Mortis-Science-Worthless-Billio...
It all comes down to incentives.
(1) As a researcher you lose funding m if you don’t produce
(2) funds aren’t allocated to reproduce
(3) Researchers who publish will block research that disagrees with their work (as they’re also reviewers) (will lose future funding / have more competition)
(4) Researchers wont rescind their work if later findings warrant it (no incentive to)
(5) N numbers are way too low (higher N is more money)
Maybe that's the case in your area of research. In mine (math, physics) it definitely isn't. So I would be a bit more careful about the wording here. ("80% of science" – what science?)
P < 0.05 is outdated at this point, for anything truly ground breaking p < 0.005 or < 0.0005 is probably a better choice, and even then I would ask "Where did you get your dataset from and did you combine (!!!) datasets from multiple orgs."
One factor contributing to this. In natural sciences, you take other people's papers as truth and build on that. In theoretical physics on the other hand the _first step_ is you reproduce their results.
As for physics, well it depends. Laboratory physics has produced the finest predictions in any science by several orders of magnitude. Quantum Electrodynamics is freakishly accurate. On the other hand it’s hard for me to see cosmology, to gently pick on an easy target, as more than extremely well researched and plausible science fiction. Then you have particle physics which has excellent laboratory equipment and produced fantastic results, but which has, in the opinion of at least one elite particle physicist personally known to me, perhaps painted itself into a corner. The Standard Model is good enough for government work, but nobody believes it’s the best possible theory.
For physics, it depends.. is it just "applied math" (so, verified easily), is it a CERN-type (LHC,...) experiment (hard to replicate, unless well.. you work at CERN), where many many people process the data, or is it something that is done only in your "lab", and hard for others to replicate.
On the other hand, finding thousands of patients and running a study is practically always hard and expensive.
But after becoming a scientist, coming across so many fraudulent papers, so much non-reproducible and poorly done research. And seeing how it's affected my own, the thing that's dearest to me, having to build on top of those results and work in that global environment. It's been heart breaking, I feel no love for science anymore.
I spend most of my career reanalyzing others' data and combining it together into larger datasets, in molecular biology and genomics. The only time I encountered mistakes it was from my own labeling errors or bugs, pre-publication. And in pre-publications datasets I would sometimes detect accidental swaps on the labels of samples as part of QC checks.
The more money is involved, the more fraud to expect. This should not surprise: fraud goes where the money is. And where there is money, the stakes are higher.
But I have seen reprehensible behavior even in opponents of the Younger Dryas Impact Hypothesis, where practically no money seemed to be at stake.
To that end, does anyone know of any list of techniques for spotting or identifying fraud in scientific works? This is a subject that I wish somebody would write a book on, so I am curious as to what techniques people use for this purpose.
There is no question that a very significant portion is wrong and some of it is fraud, but I feel like this is the wrong summary takeaway. Science isn’t the reason that science is messed up, people are the reason. All fields of human endeavor suffer from the same problems due to emotional and political and selfish people. It’s more like, welcome to humanity. Science is actually the best thing we’ve got, there is no alternative that has less BS and more truth. The 10% or so of science that’s right has transformed the earth in the last century.
I worry about framing this the right way, about the subtleties of how you say it, because there is currently a war being waged on public trust in science, and to some degree that anti-science war is being won. It’s potentially damaging to say “most of science is wrong” and just stop there. That’s a misleading framing in my opinion. In order to fix the funding problems, society as a whole needs to have trust in science, to believe that the majority of people doing science are politically impartial and also not wasting money or lining their own pockets, to believe that scientific progress is human progress.
It’s important to note that the incentive problems you cite mostly aren’t caused my any malicious intent. Disagreeing with someone else’s research, in my experience, isn’t often done with the primary goal of holding back good research, it’s done because the researchers actually disagree on the science, and the reviewer actually believes the proposed paper isn’t complete or correct or up to publication standards. Moreover, for science it’s very important for researchers to be critical of each other. That is part of why we need more replication study.
A lot of things look like science and use the word “science” in them but don’t use the process of science to make their claims to truth. This is useful because how can you argue with “the science”?
Trying to manipulate people like this is guaranteed to backfire long term. People will realize that it's being done and trust things coming from the world of science even less. Instead we need to acknowledge that there are real problems in how we practice and communicate about science and take visible steps to fix them.
Science is losing the battle for public trust because it wants to simultaneously be an infallible source of truth and this messy, chaotic discipline where we tumble towards an approximate answer. It gets defined as a one or the other when it's convenient.
In the first breath: Oh, X% of all published papers are wrong? No big deal, that's just how science works. Can we have another 100 billion of taxpayer dollars please?
In the second breath: The Science Says vaccines are safe and effective. Take it or get fired.
Even $5 to someone with a long history of finding scientific fraud puts you on the right side of history: https://www.patreon.com/elisabethbik
I’ve been donating for years and I’ve never been more proud of that decision.
That way the only possible competition will be number of citations. The pressure will be for quality, not quantity.
Well, popularity, anyway. While popularity is sometimes a useful proxy for quality, that's not always the case.
It’s probably about the worst in any field dominated by studies and statistics. Harder sciences are harder to fake as bad results can be more conclusively falsified.
It’s quite useful to listen to Buffet and Munger. They’ve accumulated lots of wisdom.
“You get what you reward for” https://youtu.be/hJYLJRr3hEY
This hour of Munger is well worth it:
Here’s the transcript:
I'm not sure if it is a good idea to try to optimize the scientific process for fraud resilience. In the end, we still rely on people doing the right thing most of the time.
He said something exactly the same, and his conscious wouldn't allow him to work in the field of science.
He left the field many years ago. He delivers mails now.
Then you also need a realignment of the entire academic system as well. A new professor spending 50% of their time redoing the experiments of others, especially on capital expenditures and grad student time is going to be at a severe disadvantage against others who spent 100% of their time building their own research agenda.
We'd probably need a whole new degree that's like a PhD, but whose dissertation isn't a novel extension of the field, but reproduction of other studies. Because as it is, a grad student can spend up to a decade on a Ph.D. just doing novel research. Now you want them to spend 50% of their time on reproduction? So is a Ph.D. supposed to take up to 20 years with your proposal? Or does a dissertation contain 50% less novel research?
Certainly we should have funding for reproduction studies, but it's not enough to just have a grant; you have to have a valid career path doing such work. That doesn't exist today.
Humans are not incentive-following automata, though we have incentive-following tendencies.
This is the biggest problem in my view. My work in R&D taught me that most of the time we don't produce anything. It's high risk. But it's high in rewards, often in adjacent areas not primarily the focus of the initial brief.
Surely all serious investors understand this. Research is something we do for marginal returns. It's not an "innovation factory". With things currently stacked against risk, research can only yield tepid results.
Not necessarily when the investor is an academic institution or a government.
It would appear that this isn’t really “Science” if the ideal of research is being veered away from so much.
Vested interests suffocating the process.
