2
$\begingroup$

In generalized least squares, I’ve specified a weights function that accounts for heterogeneity in residual variance that exist along the range of a covariate. The validation graphs (residuals plotted against covariate values) clearly shows that this adjustment gets rid of the heteroscedasticity, yet the AIC of this model is higher than the uncorrected model.

So my question is, when the model is incorrectly specified and the assumptions are violated, can the calculated likelihood (and therefore the AIC) be trusted? Would it not be better to use a more parametrized variance structure despite the higher AIC estimate?

$\endgroup$
0
$\begingroup$

Understand first what the AIC does: it penalizes models that use more parameters. This penalty is there to protect you from overfitting: adding parameters and complications will always improve the "look" of the validation graphs.

However it is fair to wonder if the AIC penalty is too high for your particular case. Keep also in mind that heteroskedasticity problem in a well specified OLS is simply one of efficiency: it makes the convergence to true $\beta$ slower which might not be an issue in your case.

If you are unconvinced about the AIC penalty my suggestion to you is to try out model-selection by cross-validation. Rather than penalizing by parameter count this method chooses the model that predicts better the data you leave out. It's more intuitive although a bit more hands on to implement but it might be a good way to double check on AIC selection.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.