# What is the difference between 'hypothesis testing' and 'model selection'?

In literature, both terms are often used synonymously or interwoven. I am now trying to find a clear distinction between both terms. From my point of view, a hypothesis is usually expressed via a model. So even if we test a null vs. alternative hypothesis, from my perspective we are doing model selection. Can someone give me an intuitive description of this distinction?

• By hovering the mouse over the tags, you can see the wiki excerpts for hypothesis-testing and model-selection. Despite their brevity, they seem to do a good job answering your question. – whuber Jan 6 '15 at 17:31

To me the distinction is that with hypothesis testing one is considering contrasts of model parameters and is not entertaining the thought of changing the model. For example, in ANOVA, people are smart enough not to convert a 4 degree of freedom $F$-test to a 3 d.f. $F$-test when comparing 5 groups and finding that two of the groups have similar means. People who formulating models often make the basic mistake of selecting which parameters should be in the model on the basis of statistical tests/comparisons, not realizing that this biases things (especially $\sigma^2$). The the example to which I just alluded, the unbiased estimate of $\sigma^2$ comes from the model having 5 regression parameters (overall intercept + 4 indicator variables).
2. which $X$s should be in the model
3. how each $X$ should be modeled (e.g., consideration of nonlinear terms)
• Nice question and answer. One issue is whether a non-rejection of the hypothesis should lead to adoption of the simpler model. I think not, in most situations. An exception is when one is checking a flexible model against an all-linear model and the test of nonlinearity with, say, 10 d.f., yields $P=.05$. I would feel safe in adopting the all-linear model. – Frank Harrell Jan 6 '15 at 20:40
• Sorry that should be $P=0.50$ not $P=0.05$. – Frank Harrell Jan 8 '15 at 4:09