3
$\begingroup$

For a simple example, I am fitting data with a likelihood function generated by a normal distribution. The first model is the normal distribution with two-parameters. The second competing model is the extreme value distribution with fixed variance=0.5, so there is only one parameter: the mean.

We can then compare those two models by AIC or likelihood-based discrimination tests for non-nested models such as Clarke Test and Vuong Test. Because more parameters are penalized for overfitting, it is possible that the second model is a better fit.

However, in theory, the second model is not a good model because it is an "overspecified" model in some sense. Here my "overspecified" means that the variance parameter should theoretically be a parameter rather than a constant. We should not specify it before fitting with the data without a firm reason.

The second model also unfairly avoids the penalization for having two parameters.

What is the rigorous academic term for this "over-specification"?

I've thought about over-fitting and under-fitting. They are relevant but exactly the word I am looking for.

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Not sure whether the sophisticated term you're apparently looking for exists. It's very easy, isn't it? Model 2 will do a crappy job if the fixed variance value is far enough off (avoiding penalisation won't help then). If it's (approximately) correct, for example because it is based on valid and strong background information, it can be good. However I don't see why anyone in practice would fix this for one model and leave it free for the other model. Either you have good knowledge of the variance or you don't. $\endgroup$
    – Christian Hennig
    Jan 9, 2020 at 0:57
  • 1
    $\begingroup$ Why is the avoidance of penalization "unfair"? You aren't estimating the parameter from the data, which is the point of penalization - to penalize you for adding more and more parameters to a model that are estimated using the data, and thereby getting better and better fits. $\endgroup$
    – jbowman
    Jan 15, 2020 at 16:07

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.