It was absolute hell. The key problem here is the waiting and uncertainty. You have the NIPT at 10w, but you can’t have the amniocentesis until several weeks later. When that came back fine, there were questions about whether it was a “mosaic” meaning only a small proportion of cells are effected. We were only really in the clear after the 20 week ultrasound.
That’s a lot of weeks to be consumed by wondering about whether to terminate the pregnancy, or wait it out for more information. I have a masters in bioinformatics (in genomics!) and my knowledge of stats and the science was next to useless in the face of these decisions.
I know of couples who simply couldn’t deal with this uncertainty and chose to terminate on the basis of this test alone.
Fortunately for us our child was fine and is a perfectly healthy 18 month old now, but I wouldn’t do the rare trisomy test again.
The tragedy of receiving news like this is probably fathomable, but I think it may be hard to grasp the emotional and intellectual agony of deciding whether to terminate a pregnancy based on a set of probabilities.
It breaks my heart to think that parents face this decision with erroneous data.
The point is that people who get a "positive" result on these tests are often put through terrifying levels of anxiety when there is no actual problem; this anxiety is often exacerbated because they aren't informed of the false positive rate. This clearly has a harmful emotional effect on people, and explaining the false positives in Bayesian terms, or reframing it in terms of sensitivity and specificity, doesn't undo that damage.
That potential harm needs to be explained to patients, and it needs to be weighed carefully against the potential benefits of the test (as is done for PSA tests for prostate cancer, which also have a high false positive rate). Given that potential for harm, it's not unreasonable to ask that these tests be more tightly regulated.
To quote the OP:
> In interviews, 14 patients who got false positives said the experience was agonizing. They recalled frantically researching conditions they’d never heard of, followed by sleepless nights and days hiding their bulging bellies from friends. Eight said they never received any information about the possibility of a false positive, and five recalled that their doctor treated the test results as definitive.
(Edit: clarified)
No you should not panic, anymore than you should celebrate and buy a yatch if you think you have 10% chance of winning a lottery.
The doctor should phrase it as "the test indicates you have a 10% possibility of cancer. The majority of people who test positive do not have cancer. Further testing is required to confirm or rule it out."
Going with the expected value of 10% of a horrid thing is still bad, or 10% of $100m is still $10m, is not applicable to a single non repeated event.
There is a good reason tests are described in terms of sensitivity and specificity ("if the answer in reality is yes, how often will the test say no?") rather than in terms of false positive or false negative rates ("if the test says no, how often is the true answer yes?"). The sensitivity and specificity are facts about the test which can conceptually stay constant[1] as you apply the test to different people. False positive and false negative rates do not have that property; they are facts about the group you're performing tests on just as much as they're facts about the test.
[1] This is not to say that the sensitivity and specificity of a test do stay constant as you apply the test to different populations. Often they won't. But it is a theoretical possibility, and even that isn't true for false negative rates.
The solution is fairly straightforward I think: require the test companies to report the posterior probability of the illness given the test result. Probably with some kind of uncertainty interval.
I don’t really understand what your implication is. Are you wishing you had not done the genetic tests? Suggesting you should have aborted the 20-week fetus based on unusual skull appearance? Suggesting that the meningitis was a result of some malpractice? Mad at medical professionals who are now “dropping hints” without saying something more substantive?
The problem with these tests highlighted (confusingly) by the NY Times article is that they test for extremely rare conditions but have a high enough false positive rate (by my napkin math, on order 0.05% = 1/2000 false positive rate) that they end up dramatically over-diagnosing these conditions.
That doesn’t necessarily mean the people designing the tests were lacking in “sound science” or were behaving maliciously. Their tests just need to be almost perfect to not overdiagnose rare abnormalities, and perfection in medical tests is a very high bar. Arguably we shouldn’t do tests for such rare conditions unless the false positive rate can be dropped by another order of magnitude or more, and if we do such tests communication to those who test positive should be much clearer. On the other hand, since genetic abnormalities are so life altering, even a 10% or 20% chance of an accurate positive might be grounds to abort a very early fetus.
It sounds like your specific problem was that the genetic tests did not return a positive result, even though it seems plausible that your child had some genetic abnormality. Not picking up every possible genetic condition is a different problem from the false-positive problem highlighted by the article.
It mixes up poor communication (reporting false-positive/negative rates as if posterior probabilities, & exaggerated confidence thereof), arbitrary-seeming decision thresholds (but their hyperventilating over '85% wrong' notwithstanding, many are probably too conservative, if anything, given how devastating many of these problems are, there should be more false positives to trigger additional testing, not less), costs of testing (sure why not but little is presented), tests which they claim just bad and uninformative (developed based on far too little _n_, certainly possible), implicit calls for the FDA to Do Something and ban the tests (not an iota of cost-benefit considered nor any self-reflection about whether we want the FDA involved in anything at all these days)... Sometimes in the same paragraph.
Plenty of valid stuff could be written about each issue, but they'd have to be at least 4 different articles of equivalent length to shed more light than heat.
This is true in so many areas of journalism but lately seems especially egregious in the NYT. And I don't really blame them, as the incentives for any individual reporter are just too great - having the government make a major policy change based on your article is basically the brass ring for an investigative reporter.
I basically can only use these types of articles as a jumping off point for my own research, as I usually find the moralizing conclusion the article comes to as unsupported.
Yep, this is the framing I came here looking for.
Investigative journalists live in the same asymmetrically-incentivized world as social science researchers. If the reporter had looked into the phenomenon and concluded "yeah, boring technical logic pretty much works as expected here" then there's no story.
Not that you're necessarily wrong, but how did you get that from the article? It didn't seem to me like they wanted a ban.
I doubt that hyping one side of this equation is fraud. Pushing the error in this direction seems like a good idea, anyway. If you have some weird illness, and the test comes back as a false positive, at least you'll continue to explore that possibility for a while. If it comes back as a false negative, then you'll spend a ton of time exploring alternatives which will be true negatives. Probably infuriating.
This mindset is ingrained in every doctor I speak to, but I think it's just so wrong.
Take DiGeorge syndrome. You have a 1/4000 chance of having it, and the test carries an 81% chance of a false positive. The above doctor calls this "marketing"? Foolishness. That's an incredibly useful test. The downside is small, and the upside is asymmetrically large.
We need far, far better screening for all sorts of things. Adult cancer and heart screens once a year, prenatal screening, and on. We do a good job with breast and prostate screens, but for rarer conditions our current approach of waiting for the disease to be symptomatic makes no sense. Part of that will be driving the cost down. There is so much market need for a legitimate version of Theranos and I'm glad there are some companies working on these things.
Do we? Unless I'm missing something, breast cancer is a huge killer and PSA tests are deeply imperfect. I am very much not expert in these areas.
You are supposed to treat a positive on NIPT as “there’s a chance your baby has this, need a more accurate procedure to confirm”.
It sounds like their ob gyn wasn’t able to explain results to them or they didn’t understand the probabilities. To be fair our provider didn’t even suggest tests for the disorders in the article, probably because of the false positive rates and rarity. Sounds like these extra screens shouldn’t be offered.
We have been undergoing IVF with my wife since 2019. (Covid made a huge mess of those plans...) One of our embryos tested as a possible positive (but only slightly) for aneuploidy of one chromosome.
The doctor, a veteran of IVF, looked at the results and said "my experience is that this is either a very small mosaic error, which tends to be utterly invisible in real life, or a computer artifact. I have never seen embryos with those borderline results develop any serious problems later. Things would be different if the aneuploidy signals were clear, but definitely do not discard this embryo".
This _does_, however, seem like a framing issue, more than a utility issue. If the tests are 100% accurate at detecting true positives, they're a great aid. But rather than framing the tests as a be-all, end-all source for information, why not frame them as "a test that suggests whether or not you should get other tests"? That simple wording change would save a great deal of added stress on someone starting or growing a family.
Having been on the receiving end of a false positive, I'd still do the test again for a hypothetical future pregnancy. Even though it was hell for a couple of days.
That got knocked down pretty quickly but wow a lot of folks picked up a big chunk of their lifetime radiation allowance because of that.
These tests seem to operate under a similar model, disregard the risks of unnecessary testing in return for information of limited utility that may cause material harm.
To evaluate the value of performing a diagnostic test as an intervention, you DO have to look at final actual patient outcomes at an appropriate end target which includes sending people unnecessarily down different treatment paths, including additional testing with additional risks. And most importantly is that, in fact, mere knowledge of diagnostic results has been PROVEN to cause harm in many scenarios.
Now... if a patient WANTS that test, I think it should be available. But whether or not it should be performed routinely without prompting is an appropriate question for regulatory bodies.
https://twitter.com/JohnFPfaff/status/1477382805583716353?t=...
'For a disease w a 1-in-20,000 risk, a test w a false positive rate of 1% and a false negative rate of 0%—an insanely accurate test—would identify 1 correct case and 200 false positives every time. Or would be wrong 99.5% of the time.
This isn’t “bad tests.” This is… baserates.'
*) My wife insists that it was a doctor, not an RN, but my brain won’t let me process that possibility.
They revised the tests so the first test would say Inconclusive rather than Positive, and ask them to repeat it. This saved some lives.
Maybe this a UX failure? Shouldn't the test designers present the results like this, even to doctors?
Your results are sent directly to your provider, so you can’t read the fine print yourself. And if you do get access to the results, the wording implies that a null result (not enough DNA collected) actually means you’re likely to have some disorder. In fact the wording here actually got worse in the three years between my two (healthy) births.
Ideally these companies should require genetic counseling before you take the test. Parents should understand that these tests are for screening purposes only, and that a definitive diagnosis can’t be gotten until 16-20 weeks. Unfortunately these companies have found a niche- parents wanting to know the sex and health of their children as soon as possible- and have no real reason to improve their practices.
Considering this as a UX challenge - imagine a grid of 10,000 dots (100x100).
Draw one box around the base rate - the rate at which you expect to find the problem in the population. If the base rate is 1%, then the box is 10x10 = 100 dots.
Then color in the dots for the test positive rate (not false positive, just all positive tests) False positives would be the colored dots outside the box.
Next to that, put strikes through the dots corresponding to your expected false negative rate.
That said, I do feel that pulling in abortions to the debate is specifically to trigger a set of readers. But to what aim? They have not established that the tests could be better. Just that when they say yes, they are still not perfect.
