undefined | Better HN

0 pointsslurry12y ago0 comments

Why would you blame the study authors for doing a correlation study? I know "CORRELATION IS NOT CAUSATION!!!" is the go-to takedown on the internet, but correlation studies actually can have significant value - otherwise peer-reviewed journals wouldn't publish them.

I'm also skeptical of Politifact's competence to adjudicate public health scholarship.

0 comments

9 comments · 4 top-level

Fomite12y ago· 5 in thread

Not only do they have significant value, but they're the only type of evidence you'll ever have for something like this - it's impossible, and almost certainly unethical - to run a randomized trial of keeping people from having health insurance.

Also, while "Correlation is not causation" is, as you mentioned, a tired canard on the internet, people often seem to forget that all things that do have a causal relationship with have some form of association. Association doesn't prove causation, but it's a damned fine first step, and miles above "guessing", which is what happens without evidence.

anatoly12y ago

You keep saying 'unethical' as if it's the end of the discussion. But what if our current standard of medical ethics prevent us from finding the better policies or the better treatments? Is this standard set in stone and never to be doubted?

Probably taking away insurance from people who have it as an experiment is too extreme, and not implementable anyway. But is it really that unethical to take a subset of population without insurance, give it to a random subset for some time, and observe the differences? Why? No one from the control group is prevented from getting insurance on their own (compare with the candidate drug trials, where the control group can't just buy the drug on their own).

Fomite12y ago

> You keep saying 'unethical' as if it's the end of the discussion. But what if our current standard of medical ethics prevent us from finding the better policies or the better treatments? Is this standard set in stone and never to be doubted?

No, it's not set in stone, but if you're talking about implementing studies now, you're not going to get major medical ethics reform first. You go with the system you have, and the system you have is probably going to push back pretty hard.

> No one from the control group is prevented from getting insurance on their own

This alone is a difference between the control and treatment groups that takes place post randomization, and knocks said experiment back into the realm of "correlational"

ekianjo12y ago

Sorry, but no. 100 times no. Correlation is very often linked to a third factor or multiple factors which are not visible, nor measured in observational studies. Besides, let's not disregard the fact that correlation still has some good chance to be pure luck. Even correlation with 95% confidence statistical significance can be a random result in a non-nil number of times.

So, no, you never prove anything nor imply anything at all with correlation. You're still guessing.

Fomite12y ago

Find me where I said proof.

There are all kinds of things that we cannot prove, because it is either impossible or wildly unethical to conduct a randomized study. For those things, you can make a determined effort to control for as many "third factors" - the technical term for them is confounders - and that gives you a level of evidence which is well above guessing.

Since I can't reply to your comment, my responses here:

> "You didn't say proof but you said it's better than guessing, and I don't agree with you at all."

It is better than guessing. You're welcome to disagree, but a well conducted observational study is considerably firmer evidence than pulling it from your posterior.

> "What if there is a correlation between Vegetarian-lifestyle and Serial-killers ? Does it tell you that it's better than guessing ? Do you even question if the association/correlation makes remote sense ? Is there any underlying mechanism of action that would remotely explain rationally why this correlation could be linked to any real causation phenomenon ?"

All you've done is describe a really bad study. You can have really bad RCTs as well, by the way.

Of course you question whether or not an observed association has a clear biological or social mechanism. And you attempt to control for other variables that might influence the link between your exposure and your outcome. You run followup studies in different populations to try to understand if the result is a widespread phenomena, or a fleeting bit of statistical noise.

Basically, you do your job well. Which is why I used phrases like "a good first step".

Your example is about as useful as "Programming is useless because once I coded something poorly and corrupted my data".

2 more replies

nzp12y ago

> So, no, you never prove anything nor imply anything at all with correlation. You're still guessing.

This is a totally unreasonable stance to take. You can't even imply anything at all with correlation? Really, nothing at all? It's no better than a random guess? Try actually doing some actual science with this attitude, and keep to it consistently, and let me know how far you make it. In fact, try the same thing with ordinary life, any kind of reality where your decisions have consequences in reality.

aamar12y ago

I agree that correlation studies can be valuable, but I see many correlation studies as (intentionally or not) exploiting a propensity (arguably a bug) in human reasoning that conflates it with causation. In this situation, we have thorough documentation of multiple people clearly making the error.

I'm actually not sure the study authors should be blamed.

But: given how politicized this question is, they could have reasonably anticipated the misuse of their results, and thus could have written their results in such a way as to avoid this. Or they could have publicly corrected non-experts who cited them for causation. Or: they could have controlled for factors that would make mortality and insurance-status independent. This last option is difficult & requires complex judgement calls (see [1] for a reasonable attempt), but even if you feel the other two aren't required of academics, this last one very much may be.

(Separately, I agree that Politifact should not be trusted automatically, but these seem like reasonable analyses, and I did not quickly find anything better.)

[1] http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2739025/

mattfenwick12y ago

> correlation studies actually can have significant value - otherwise peer-reviewed journals wouldn't publish them

This is an appeal to authority. And unfortunately, the evidence is accumulating that there are problems with that authority. The success of peer-review depends on the quality of the peers and of the review. If those reviewing don't understand how to evaluate a correlation study, or do understand but don't take the time to properly evaluate it, then garbage will slip through.

It turns out, lots of garbage is produced and sent to the reviewers, as was noted in a recent Nature feature:

http://www.nature.com/news/scientific-method-statistical-err...