Do I accept or reject the null hypothesis?

M1 : Y ∼ β0 + β1x1 + β2x2

M2 : Y ∼ β0 + β1x1

anova(M1,M2) shows a p-value of 0.0001, so we prefer M1 at significance level 0.05.

Would that be correct? I thought that if .0001<.05, I should reject the hypothesis M1?

Your confusion stems from the lack of clarity about the null hypothesis that is being tested. P-values should always be interpreted taking into account the null hypothesis.

When we compare two models using anova(M1, M2), we are performing a likelihood ratio test with the null hypothesis: is the extra parameter in M1 ($$\beta_2$$), when compared to compared to M2, equal to zero?

If you reject the null hypothesis when the p-value is 0.0001 < 0.05, you can state that there is enough evidence to say that the extra parameter $$\beta_2$$ in M1 is non-zero. In this way, you will prefer M1 instead of M2. Otherwise, you would miss the explanation of $$Y$$ given by $$X_2$$.

One additional detail is that we never accept a hypothesis. The absence of evidence is not evidence of absence. You can read more here. For example, if you had observed a p-value > 0.05, then you would only be able to state that there is not enough evidence that the parameter is not zero (not rejecting the null hypothesis), but you could not say that the parameter is zero (because that is accepting the null hypothesis).

• So it is wrong to say "we prefer M1 at significance level 0.05" right? Is that what you are saying?
– Mary
Commented Aug 6, 2020 at 18:54
• No. I edited my answer to give you more details about not accepting a hypothesis. You could say that you prefer M1 at 5% significance level because you reject the null hypothesis (M2 is enough) was rejected. Commented Aug 6, 2020 at 19:17
• I kind of understand what you are saying. However, why do we "prefer M1" and not M2? Is it because we can't accept a hypothesis plus the null hypothesis being unclear? Is that what's making the difference?
– Mary
Commented Aug 7, 2020 at 0:39
• You prefer "M1" because you rejected that $\beta_2$ is equal to zero, therefore the relationship between $Y$ and $X_2$ is important. Commented Aug 10, 2020 at 7:14