7
$\begingroup$

Normally when somebody finds an association in an epidemiological study people are quick to point out that it doesn't prove causality, that there are problems of missing co-founders, that it is at best hypothesis generating and at worst spurious. This leads to people not putting much weight on associations found in epidemiological studies.

What if it goes the other way around? Say I already have a theory, maybe with some small earlier studies to back it up and even a good theoretical explanation for the effect. Then I do a big, well powered, epidemiological study and fail to find the association the theory predicts. How much weight can I put on the result now?

Intuitively it seems to me that the result would be quite damning, despite it being only a epidemiological study. But I have long since learned not to trust my intuition when it comes to statistics.

Do all the weaknesses of associations found in epidemiological studies also apply to when you fail to find an association?

$\endgroup$
5
  • 1
    $\begingroup$ Correlation doesn't imply causation, but causation does imply correlation. Taking the contrapositive, lack of correlation implies lack of causation. So this removes one of the criticisms commonly levelled at epidemiological studies - other criticisms may still apply, however. $\endgroup$ Jun 21 '12 at 11:46
  • 2
    $\begingroup$ @ChrisTaylor, assuming you're using the word "correlation" in the usual statistical sense, causation does not need to imply correlation if the relationship is non-linear. If $X \sim N(0,1)$, then $X$ is causally linked to $|X|$ but not correlated. $\endgroup$
    – Macro
    Jun 21 '12 at 12:12
  • 1
    $\begingroup$ @Macro CV keeps going back and forth on this - and technically, anyone using 'correlation doesn't imply causation' in regards to an epidemiological effect estimate is also wrong by the same logic. But oddly, it doesn't seem to get pointed out as much then. But causation does imply association. $\endgroup$
    – Fomite
    Jun 21 '12 at 18:26
  • $\begingroup$ @EpiGrad, re: and technically, anyone using 'correlation doesn't imply causation' ... is also wrong by the same logic. - I probably don't disagree but I'm not sure what you're saying. Using the usual statistical definition of correlation, it clearly doesn't imply causation. Re: "But causation does imply association." - yes, of course. Otherwise, I'm not quite sure what "causation" would even mean :) $\endgroup$
    – Macro
    Jun 21 '12 at 18:57
  • $\begingroup$ @Macro The "Yes, of course" is something a remarkable amount of the people who trot out that particular statement seem to miss. $\endgroup$
    – Fomite
    Jun 21 '12 at 18:59
3
$\begingroup$

Yes, the weaknesses of associations found in epidemiological studies also apply to a failure to find an association. You've already eliminated the first go-to problem, that of a study being underpowered, so at the moment we're just talking about bias. Two issues that may mean your study is failing to find a true association:

  • Confounding. There are conditions where a confounding variable will drive a result toward the null. For positive effects, this is the confounding variable being negatively associated with both the exposure and the outcome. For negative effects, the reverse. This could easily drive, depending on the strength of the confounding, a real relationship downward enough that you cannot find it.
  • Misclassification. Non-differential misclassification is when everyone in your study has an equal probability of being in the wrong category. This tends to drive estimates toward the null. Differential misclassification, where particular categories are more prone to being misclassified, can drive results toward or away from the null.

So no, the results of a single study should never be taken as definitive "proof", one way or the other, of a causal relationship.

$\endgroup$
2
  • $\begingroup$ In the case of confounding is the probability of having such a confounding variable that drives the association towards null as big as the probability of a confounding variable producing a false relationship? $\endgroup$
    – Mr Alpha
    Jun 30 '12 at 16:49
  • $\begingroup$ @MrAlpha That depends on how your variable works as a confounder, and your effect estimate to begin with. But it is not an uncommon occurrence, and should always be considered. $\endgroup$
    – Fomite
    Jul 1 '12 at 19:37
1
$\begingroup$

I think it's worth distinguishing a few aspects of the problem:

  • Precision: If you have a bigger sample size, you will typically be able to estimate parameters with greater precision whether you define this in a frequentist sense in terms of smaller confidence intervals or in a Bayesian sense in terms of smaller credibility intervals. Thus, if you conduct an epidemiological observational study with a bigger sample you will have greater precision in describing the magnitude of a parameter of interest. This is true whether the parameter is a simple correlation coefficient or a regression coefficient in a broader model with many other predictors. It's also true whether the parameter of interest is derived from an observational study or an experimental study. So for example, you might get a very accurate estimate of the correlation between eating chocolate and Body Mass Index. Associations, whether causal or not, are real and interesting.

  • Generalisation: However, even if you know the value of a parameter in your particular sample, there are still issues of generalisation. In epidemiology there are plenty of issues related to generalising across time, culture, social groups, and so on. Often we have theories and empirical evidence to guide us in this process of generalising. For example, we may argue that is safe to generalise a chocolate-BMI association over reasonable periods of time, but that perhaps across nations it is more complex, perhaps because of different eating and exercise habits, etc.

  • Causality versus association: However, you seem to be particular interested in causal inference. At a basic level, the absence of an association in an observational study does not prove the absence of causality, just as the presence of an association in an observational study does not prove causality. Even if observational studies showed no relationship between, for example, chocolate and BMI, this would not prevent experimental studies from showing that when kids were fed more chocolate, they put on weight. The association or lack of association in an observational study may be informative as to causal processes, but it is not definitive. You still need to think hard about the theorised underlying causal processes.

$\endgroup$
0
$\begingroup$

Well, in physics we are very used to what you are saying and, to me at least, it is in fact far more exciting when this happens that when some researcher confirms theory. In these cases you just report that: you've found no evidence of what the theory predicts.

After checking your results and the possibe flaws of your study, it is also good practice to elaborate (at least qualitatively) your own guesses about why theory fails in your case.

$\endgroup$
2
  • $\begingroup$ Physical theories usually make specific quantitative predictions. These serve as null hypotheses. Your case of "no evidence" is often actually a case of rejecting the null. The present question asks about what can be concluded when one fails to reject the null. $\endgroup$
    – whuber
    Jun 22 '12 at 13:31
  • $\begingroup$ Right. Also, I thought I posted this as a comment not as an answer! $\endgroup$
    – Néstor
    Jun 23 '12 at 11:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.