# Theoretical objections to hypothesis testing [duplicate]

We've all seen how $p$-values can be misinterpreted to draw false inferences. However, I'm interested in learning more about theoretical/mathematical/philosophical objections to the hypothesis-testing-paradigm, ie. I'd like to learn why someone would prefer (say) a likelihood ratio over a hypothesis test.

I've read some articles by Andrew Gelman on this topic, but I'm trying to find more literature. Any help would be greatly appreciated.

• Couple of books that you might be interested in "What if there were no significance tests?", edited by Harlow, Mulaik and Steiger, and "The Cult of Statistical Significance: How the Standard Error Costs Us Jobs, Justice, and Lives" by Ziliak and McCloskey. – Jeremy Miles Mar 26 '13 at 15:14
• This is just my own opinion, but too many people confuse statistical hypotheses with scientific ones. Even if you have "rejected the null hypothesis" that some statistical parameter = 0, it doesn't necessarily mean you've even asked a scientific question (from a philosophical, epistemological point of view). – D L Dahly Mar 26 '13 at 19:04
• – Michael Bishop Mar 26 '13 at 19:57

## 3 Answers

Well, try virtually any book about Bayesian statistics. I haven't find one that doesn't have at least a few paragraphs debunking practice of significance testing. Really. And Gelman's books are good example of that. I used to remember relevant titles (I still remember my first one: E.T. Jaynes "Probability theory - the logic of science". Jaynes definitely wrote with passion, and the controversy over significance testing is something, that really made me do statistics professionally ;-) ), but soon I got overwhelmed by the abundance of literature on this topic.

Try also the phrase +bayesian significance testing fisher in Google Scholar and I guarantee you will find at least dozen good references (and some interesting info too).

This paper characterizes the publication/scientific practice in psychology as being a ritual of finding p<.05. They argue against hypothesis testing without solid theoretical basis. Note, though, that it is--if at all--more philosophical than anything else. But maybe it helps.

I'm reading an article right now called "Principles of Inference and Their Consequences" by D.G. Mayo and M. Kruse. It's a good article so far about how hypothesis testing can violate principles like the likelihood principle(LP). They go through a concrete example of coin-tossing and show how the concept of statistical significance violates the LP.