Skip to main content
1 of 2
Ben Bolker
  • 47.4k
  • 3
  • 131
  • 182

For AIC/BIC selection it doesn't really matter whether you choose to count the variance parameter, as long as you are consistent across models, because inference based on information-theoretic criteria only depends on the differences between the values for different models. Thus, adding 2 (for an additional parameter) across the board doesn't change any of the delta-*IC values, which is all you are using to choose models (or do model averaging, compute model weights, etc. etc.).

(However, you do have to be careful if you're going to compare models fitted with different procedures or different software packages, because they may count parameters in different ways.)

It does matter if you are going to use AICc or some other finite-size-corrected criterion, because then the residual information in the data set is used (the denominator of the correction term is $n-k-1$). Then the question you have to ask is whether a nuisance parameter such as the residual variance, which can be computed from the residuals without modifying the estimation procedure, should be included. I wrote in this r-sig-mixed-models post that I'm not sure about the right procedure here. However, looking quickly at Hurvich and Tsai's original paper (Hurvich, Clifford M., and Chih-Ling Tsai. 1989. “Regression and Time Series Model Selection in Small Samples.” Biometrika 76 (2) (June 1): 297–307, doi:10.1093/biomet/76.2.297, http://biomet.oxfordjournals.org/content/76/2/297.abstract), it does appear that they include the variance parameter, i.e. they use $k=m+1$ for a linear model with $m$ linear coefficients ...

Ben Bolker
  • 47.4k
  • 3
  • 131
  • 182