What is your favorite statistical quote?
This is community wiki, so please one quote per answer.
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.Sign up to join this community
"If you put a buttock on a hot plate and another one on an ice cube, the average is good, but in reality your bottom is in trouble."
Not yet famous, but could become so.
"If a problem can not be tackled nonparametrically, it is dangerous to tackle it parametrically. But on the other hand, if it can be tackled nonparametrically, it would be better to tackle it parametrically." -- Sir David Cox
The Median Isn't the Message
CauseWeb has a collection of statistics quotations. Many have already been repeated here, but it has plenty that haven't yet been quoted, such as
"The only statistics you can trust are those you falsified yourself."
(Falsely attributed to Sir Winston Churchill.) For the rest, follow the CauseWeb links to Resources->Fun->Quote.
"Winwood Reade is good upon the subject. He remarks that, while the individual man is an insoluble puzzle, in the aggregate he becomes a mathematical certainty. You can, for example, never foretell what any one man will do, but you can say with precision what an average number will be up to. Individuals vary, but percentages remain constant. So says the statistician".
(Sherlock Holmes speaking to Dr. Watson in Arthur Conan Doyle's "The Sign of the Four")
Good statistics involves principled argument that conveys an interesting and credible point.
-- Robert P. Abelson, (1995) "Statistics as Principled Argument"
We left in our mathematical model a gap for the exercise of a more intuitive process of personal judgement
-- Egon Pearson, quoted in Abelson (1995).
A table without stars is like champagne without bubbles! - David Giles
Do not make things easy for yourself by speaking or thinking of data as if they were different from what they are; and do not go off from facing data as they are, to amuse your imagination by wishing they were different from what they are. Such wishing is pure waste of nerve force, weakens your intellectual power, and gets you into habits of mental confusion.
--Mary Everest Boole
Check out "Statistician's Blues" by Todd Snider who is an alternative-country singer-songwriter. Warning, if you are sensitive to "bad" words, don't listen to the song. If you have a good or perhaps twisted sense of humor you will enjoy.
[T]he p-value is the probability of obtaining data at least as extreme as the ones observed, if the null hypothesis is true. This is a world apart from saying that it is the probability of the null hypothesis being true, given that you observed that extreme data! Beware! If your ability on the long jump puts you in the 99.99% percentile, that does not mean that you are a kangaroo, and neither can one infer that the probability that you belong to the human race is 0.01%. - Tomasso Dorigo
A variation on the Fisher quotation given here is
Hiring a statistician after the data have been collected is like hiring a physician when your patient is in the morgue. He may be able to tell you what went wrong, but he is unlikely to be able to fix it.
But I heard this attributed to Box, not Fisher.
Perhaps not overly famous among statisticians but reduced-form econometricians will know it well:
If you can't see the causal relation of interest in the reduced form, it's probably not there.
An intense preoccupation with the latest minutiae and indifference to the social and intellectual forces of tradition and revolutionary change, combine to produce the Mandarinism that some would now say already characterizes academic statistical theory and is most likely to describe its immediate future.
The statisticians of the past came into the subject from other fields - astronomy, pure mathematics, genetics, agronomy, economics etc. — and created their statistical methodology with a background of training for a specific scientific discipline and a feeling for its current needs. So for the future I recommend we work on interesting problems and avoid dogmatism.
-- Herbert Robbins in "Wither Mathematical Statistics?" as quoted by Peter J. Huber in "Speculations on the Path of Statistics".
Robbins, Herbert. "Wither Mathematical Statistics?" Advances in Applied Probability 7 (1975): 116-21. doi:10.2307/1426316.
Brillinger, D. R., L. T. Fernholz, and S. Morgenthaler, eds. The Practice of Data Analysis: Essays in Honor of John W. Tukey. Princeton University Press, 1997. http://www.jstor.org/stable/j.ctt7zthdd.
In his Fisher lecture, Box defines mathematistry as
the development of theory for theory's sake, which, since it seldom touches down with practice, has a tendency to redefine the problem rather than solve it. Typically, there has once been a statistical problem with scientific relevance but this has long since been lost sight of.
He also defines cookbookery as
The tendency to force all problems into the molds of one or two routine techniques, insufficient thought being given to the real objectives of the investigation or to the relevance of the assumptions implied by the imposed methods.
Box, G. E. P. 1976. Science and Statistics. Journal of the American Statistical Association, 71: 791–799.
Roderick J. Little (2013) In Praise of Simplicity not Mathematistry! Ten Simple Powerful Ideas for the Statistical Scientist, Journal of the American Statistical Association
A statistical procedure is not an automatic, mechanical truth-generating machine
This is a two part quote I've heard a few times:
Machine learning is statistics minus any checking of models and assumptions.
Brian D. Ripley
In that case, maybe we should get rid of checking of models and assumptions more often. Then maybe we'd be able to solve some of the problems that the machine learning people can solve but we can't!
Link (Ripley is quoted as making the statement at the useR forum, Gelman responds to the quote on his blog).
There's some irony in the sides taken by these two statisticians in this conversation. That is, some of Brian Ripley's early work was on Neural Networks, which is an extremely popular topic in Machine Learning, where as Andrew Gelman is exclusively known for Bayesian statistics in which he advocates taking one's statistical models very seriously.
The Government are very keen on amassing statistics—they collect them, add them, raise them to the nth power, take the cube root and prepare wonderful diagrams. But what you must never forget is that every one of those figures comes in the first instance from the chowkidar (village watchman), who just puts down what he damn pleases [link].
-- Josiah Stamp, recounting a story from Harold Cox, Some Economic Factors in Modern Life (1929), p. 258.