# Statistics, war stories, data intuition

I think it is fair to say statistics is an applied science so when averages and standard deviations are calculated it is because someone is looking to make some decisions based on those numbers. Part of being a good statistician then I would hope is being able to "sense" when the sample data can be trusted and when some statistical test is completely misrepresenting the true data we're interested in. Being a programmer that is interested in analysis of big data sets I'm relearning some statistics and probability theory but I can't shake this nagging feeling that all the books I've looked at are kind of like politicians that get up on stage and say a whole bunch of things and then append the following disclaimer at the end of their speech

Now, I'm not saying that this is good or bad but the numbers say it's good so you should vote for me anyway.

Maybe you get that but maybe you don't so here's a question. Where do I go to find war stories by statisticians where some decisions were based on some statistical information that later turned out to be completely wrong?

-

This isn't exactly what you're asking for, but I think you'd like David Freedman's work. He calls BS on misapplication of statistical tests, etc... See here: http://www.stat.berkeley.edu/~freedman/. One of my favorites is “What is the probability of an earthquake?”.

-
Actually this is pretty close to the kind of stuff I'm looking for. – davidk01 Jan 2 '11 at 0:12

You might check out a recent presentation on SSRN by Bernard Black, "Bloopers: How (Mostly) Smart People Get Causal Inference Wrong." http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1663404

I will say that I also admire David Freedman and appreciate his work. Though I was a UC Berkeley grad student while he was here, he passed away before I had a chance to take his course. You might have a look at his collected works edited by a few other Berkeley professors: "Statistical Models and Causal Inference: A Dialogue with the Social Sciences."

Charlie

-