In literature, both terms are often used synonymously or interwoven. I am now trying to find a clear distinction between both terms. From my point of view, a hypothesis is usually expressed via a model. So even if we test a null vs. alternative hypothesis, from my perspective we are doing model selection. Can someone give me an intuitive description of this distinction?
To me the distinction is that with hypothesis testing one is considering contrasts of model parameters and is not entertaining the thought of changing the model. For example, in ANOVA, people are smart enough not to convert a 4 degree of freedom $F$-test to a 3 d.f. $F$-test when comparing 5 groups and finding that two of the groups have similar means. People who formulating models often make the basic mistake of selecting which parameters should be in the model on the basis of statistical tests/comparisons, not realizing that this biases things (especially $\sigma^2$). The the example to which I just alluded, the unbiased estimate of $\sigma^2$ comes from the model having 5 regression parameters (overall intercept + 4 indicator variables).
Model selection often involves (dangerously) choosing
- among a set of competing model families or distributions
- which $X$s should be in the model
- how each $X$ should be modeled (e.g., consideration of nonlinear terms)