14
$\begingroup$

I am drafting a literature review on a current public health issue where data are confounded:

What are common historical case-studies that are used in public health/epidemiology education where invalid or confounded relationships or inferences were intentionally or erroneously employed in public health policy and legislation?

The automobile fatality surge of the 1960s and subsequent evidence-based, government-led study which determined seatbelts and eventually airbags should be required by law is a great example of HOW public health policy should be driven by statistically powerful inferences and models.

I am looking more for examples of cases of the opposite type (bad science to make policy hastily). However, if nothing else I would like to learn of more cases similar to the previous example of powerful studies for successful public health benefit.

I want to use these as examples to demonstrate how evidence-based statistical public health research is important to the making of policy.

$\endgroup$
  • $\begingroup$ @Nick Cox - thanks for helping to make this a better question. Would love to have you weigh in, if even from an environmental standpoint - I think the more examples the better. $\endgroup$ – cerd Aug 12 '13 at 9:02
  • $\begingroup$ You're welcome, but my contribution here will be no more than editorial. $\endgroup$ – Nick Cox Aug 12 '13 at 9:08
  • 1
    $\begingroup$ This is a nice paper that discusses some issues: crab.rutgers.edu/~goertzel/mythsofmurder.htm (and it has a great title) $\endgroup$ – Jeremy Miles Aug 13 '13 at 20:29
7
$\begingroup$

I think the best example of this may likely be the controversy around hormone replacement therapy and cardiovascular risk - large cohort epidemiological studies seem to suggest a protective effect and health policy and physician recommendations were made on this information.

Follow-up RCTs then seem to show that there's actually an increased risk of myocardial infarction in women placed on HRT.

This goes back and forth for a bit, and has been used as one of the canonical cases to attack epidemiology as a field, but a recent re-analysis by Hernan seems to propose that the two studies actually don't have discordant results if you make sure you ask the same question.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ very fascinating. In the course of this specific controversy playing out, has the FDA weighed in at all from a risk evaluation/mitigation standpoint? $\endgroup$ – cerd Aug 12 '13 at 8:46
  • 2
    $\begingroup$ +1 Very interesting example! It's also very nice to see someone mentioning Miguel Hernan. As a side note: His causal inference book is (still) freely available. $\endgroup$ – COOLSerdash Aug 12 '13 at 12:20
3
$\begingroup$

A really interesting example I personally like is taken from the book Freakonomics by Steven D. Levitt and Stephen J. Dubner. There is a chapter in the book that discusses correlation vs. causality. Correlation between two statistical variables does not necessarily imply that these variables are statistically dependent, but a mistake along these lines was made by experts. Quoting from the book:

"A tricky beast, Polio was extremely difficult for researchers to pin down. They couldn’t figure out how it was passed or when/how it expressed itself. We have a tendency to remember this time as one in which Polio was ‘epidemic’ when, in fact, it was not affecting large swaths of the population (compared with the more common measles, for example). The reason it was seen as epidemic was because it was so frightening.

What researchers DID manage to determine in their studies was that Polio infection rates went UP in the Summer. They also saw that ICE CREAM CONSUMPTION went up in the Summer. And so they concluded that ice cream consumption led to Polio and for a time ice cream was demonized. "

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Details of how this idea was "employed in public health policy and legislation"? $\endgroup$ – Nick Cox Aug 12 '13 at 7:28
  • $\begingroup$ Eating ice cream also urge to do murder $\endgroup$ – ttnphns Aug 12 '13 at 7:45
  • $\begingroup$ @Nick Cox , this research was published in The American Journal of Pathology which affects public health policy $\endgroup$ – Leeor Aug 12 '13 at 7:50
  • $\begingroup$ @Leeor That's not convincing in itself. I don't think this question is well served just by examples of misinterpreted correlations. It has more bite than that. $\endgroup$ – Nick Cox Aug 12 '13 at 8:05
  • $\begingroup$ @Nick Cox I disagree. The question is meant to "demonstrate how evidence-based statistical public health research is important to the making of policy". My answer directly relates to statistical public health research. $\endgroup$ – Leeor Aug 12 '13 at 8:29
3
$\begingroup$

In his paper, "Statistical Models and Shoe Leather" (1991), David Freedman presents some cautionary tales in epidemiological studies. He offers Snow's analysis of cholera in London as a success, not due to statistical modeling, but rather due to diligent data collection. Here's the abstract:

Regression models have been used in the social sciences at least since 1899, when Yule published a paper on the causes of pauperism. Regression models are now used to make causal arguments in a wide variety of applications, and it is perhaps time to evaluate the results. No definitive answers can be given, but this paper takes a rather negative view. Snow's work on cholera is presented as a success story for scientific reasoning based on nonexperimental data. Failure stories are also discussed, and comparisons may provide some insight. In particular, this paper suggests that statistical technique can seldom be an adequate substitute for good design, relevant data, and testing predictions against reality in a variety of settings.

Sociological Methodology. 21: 291-313.

| cite | improve this answer | |
$\endgroup$
3
$\begingroup$

The case of high-dose chemotherapy with bone-marrow-transplant rescue as treatment for advanced breast cancer in the 1990's is one such instance. A series of low-quality studies were used to push through legislation mandating health insurer coverage in some states. When the large randomized trials were completed, there was no measurable benefit.

http://www.gao.gov/products/HEHS-96-83

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.