Causation implication

Question

I recently read an article about how you can increase longevity by sleeping less. This article, like many others I've read, references a statistical study and implies that causation was found between two events(sleeping less and living longer). Well I'm not a statistician, but I'm aware of the common fallacies about causation and I find it very hard to accept the information given in this article and other similar ones. Even further, I can almost never know from an article if the original study found causation or not, without taking the time and digging into it.

Although I'd really like to, I'm not going to go into any depth bringing examples of how the causation might be wrong in the given article, because I would most likely be preaching to the choir. Luckily though, to my relief, the original article in NY Times only talks about correlation and even has a disclaimer in the end that basically says that the study doesn't imply causation.

Also, I now read that, in accordance with my intuition, it is actually really hard to find and/or test for causation. Reading through the wiki page on the subject I saw two models which seemed to be the most common ones used to find actual links in causation: Granger causality test and convergent cross mapping. But I see, that both of these have some gaps in them.

Now, I have a few questions regarding all this.

Q1: Are there any models that can, with high accuracy, find actual causation in such widespread studies? If so what are the most common ones?

Q2: How often do these widespread studies actually test for causation? E.g. if I read somewhere that a very broad study implies that A causes B, do I have any reason whatsoever to believe that there's something more than correlation behind those claims.

Q3: Is there a good technique that can be used by a non-statistician to filter out good statistics from the bad ones in everyday life.

Answer 1

Starting with Q2: convincing readers that a causal effect has been found is often the overriding concern of research articles in social sciences, psychology and medicine. However this usually cannot be done through a simple test but by proposing a causal theory, presenting evidence consistent with the theory in the form of correlations and other statistical patterns, pre-empting alternative explanations, and presenting evidence that is not consistent with the alternative explanations.

(Q1) Regression analysis allowing the researcher to examine the conditional correlation between an explanatory variable and a dependent variable while controlling for other possible explanatory variables is still probably the most common method. This does not by itself demonstrate causation between the variables of interest, but helps to isolate and reject competing causal explanations. In particular, it is difficult to argue that there is a causal relationship when there is no correlation after controlling for other variables (although when the explanatory variables are strongly correlated with each other, even this can be questioned).

Some other common methods that try to improve on basic regression analysis for eliminating competing explanations of hypothesised causal effects include

• 'natural experiments', for instance where one region of a country has benefited from a new policy but another, otherwise similar, region has not
• matching techniques such as propensity score matching, which try to control specifically for the fact that the 'treatment' group (e.g. people who sleep less) in an observational study may differ systematically from the 'control' group (people who sleep more) in ways that affect the outcome of interest (longevity)

(Q3) Arguably there is no simple technique for filtering out good statistics; it depends as much on knowledge of the field as knowledge of the statistical methods. Assuming an article shows a correlation between two variables, consider what the alternative explanations for that correlation could be (e.g. an unobserved variable influencing both, or reverse causation) and whether the article at least tries to eliminate those alternative explanations. There will nearly always be some form of regression or matching analysis to control for other variables; if nothing like that is mentioned in the abstract then a causal conclusion is likely to be unsound (but such articles would rarely get published in academic journals).

Also consider the level of generality claimed for the causal effect. E.g. the claim that less sleep leads to more longevity in abstract terms sounds like it is suggesting a universal effect, applicable to humans in all places and times. To make a general claim like that plausible would depend on subject knowledge (e.g. prior evidence that the determinants of longevity are similar across countries) and a strong argument that the sample is representative (e.g. the article describes how they were selected and discusses how potential sample bias was avoided).

For me the most convincing evidence from observational studies is when a series of studies by different researchers, using different datasets and techniques, have consistently found evidence that does not contradict the proffered causal explanation, and progressively eliminated alternative explanations.

Answer 2

Q2: As far as I know there are 2 good ways to test for causality. It is done by the design of the study. Two designs work here:

• In a situation where you can control ALL independent variables (usually in experiments). This makes sure that A is causing B, and there is no C causing the effect (and also not the reverse effect; B causing A).
• In longitudinal studies.

Q1: Rubin's Causal Model: http://en.wikipedia.org/wiki/Rubin_causal_model

Q3: There is no "trick" to seeing whether the statistics of one single article are good. I find the most important questions you should ask yourself:

• Who did the research?
• What is to be gained by the researcher with a favorable outcome?