Skip to main content
1 of 4
Sextus Empiricus
  • 86.6k
  • 6
  • 115
  • 304

AIC is more a model selection method in which you do not favour some null hypothesis.

Contrary to this, with a hypothesis test (and with p-values) you choose to reject or not reject a null hypothesis based on whether some (subjective) significance level is being obtained or not. The hypothesis test generally favours the null hypothesis. With a high significance level requirement you can choose to not reject the null hypothesis and not select a more complex model, even when the more complex model is improving the likelihood by a lot.

The selection based on AIC is very much related to a Wilks' hypothesis test (which approximates the distribution of the likelihood ratio by a chi squared distribution). When you compare a full model with a model with one parameter less then the difference in AIC is related to the log likelihood ratio

$$\lambda_{LR} = - 2 \log \left(\frac{L_0}{L_1}\right) = -2\left( \log L_0 - \log L_1 \right) = AIC_1 - AIC_2 -2 \sim \chi^2 $$

Where here the $\sim$ means assymptoticaly equal in distribution (more often it means 'is distributed as').

For instance when we compare the full model (AIC = 519.54) with the model without the cement variable (AIC = 535.27) then the log likelihood ratio is 11.73 and this corresponds to a p-value of 0.0002110562 much similar to the p-value of your ANOVA test which is 0.0002355.

When you select a model based on the lowest AIC then you are effectively using a boundary value for the log likelihood value of 2 (if the difference is only one parameter) and this corresponds to a p-value of 0.157, much less strict than most significance values like 0.05, 0.01 or smaller.


Sidenote. The reason that you ended up removing the I(water^2) term is slightly different. It is because the ANOVA is computed by sequentially removing the terms. You have been comparing the removal of I(water^2) after first already removing Water:Coarse.Aggregate.

Sextus Empiricus
  • 86.6k
  • 6
  • 115
  • 304