I suspect that most users of statistical tools are ancillary users (folks who have had little to no formal training in statistics). It’s very tempting for researchers and other professionals to apply statistical methods to their data simply because they have seen it “done before” in peer-reviewed papers, grey literature, the web or at a conference. However, doing so without a clear understanding of the required assumptions and the statistical tool’s limitations can lead to erroneous results—errors often unacknowledged!
I find that undergraduate students (particularly in the social and natural sciences) are either unaware of the statistical pitfalls or find these pitfalls inconsequential (the latter being most often the case). Though examples of improper use of statistical tools can be found in many introductory level text books, the web or StackExchange, I have a difficult time finding real-world examples that have had detrimental results (e.g. cost in $, lives impacted and careers lost). To that end, I am looking for real-world examples that highlight the misuse of statistical methods for which:
- the statistical methods used are typically covered in introductory stats courses (i.e. inferential stats, regressions, etc…)
- the end result have had costly consequences (dollars lost, lives impacted, careers shattered etc…)
- the data are readily available for use as working examples in a course (the purpose is to have students work through real world examples that have had real world consequences.)
One non-statistical example I like to bring up to students when discussing the importance of properly defining the units in a research project is the “metric mishap” that led to the loss of a $125M satellite! This usually invokes an :-o factor from the students and seems to have a lasting impression (at least throughout their short academic lives).