# Thoughts on model self-penalization amidst difficult parameter estimation

It is well accepted that one should account for model complexity when performing model comparisons, and the general procedure is to penalize more complex models more strongly. While this makes sense when the parameters of a given model are easily estimated (i.e. analytically, as with the mean, variance, etc), it occurs to me that if parameter estimation is a more difficult endeavour then more complex models may to some degree self-penalize. That is, if parameter estimation requires search of a parameter space, presumably larger parameter spaces are more difficult to search and therefore any finite search algorithm is more likely (as the parameter space expands) to terminate prior to finding the point of global maximum likelihood.

Has this idea been considered in the statistical literature at all?

• It sounds like the Occam's razor. – user10525 Apr 22 '12 at 9:18
• @Procrastinator As far as I understand Mike, he is less concerned with complexity per se but with the parctical costs of complexity due to more difficulties in finding the optimal parameter estimates. – Henrik Apr 22 '12 at 11:10
• @Henrik That is exactly why Occam's razor jumps on the stage, because it is related to the principle of parsimony. – user10525 Apr 22 '12 at 11:20
• @Procrastinator But the principle of parsimony is totally unrelated with the principle of how to obtain the optimal parameter estimates. – Henrik Apr 22 '12 at 12:54
• @Henrik The OP says "it occurs to me that if parameter estimation is a more difficult endeavour then more complex models may to some degree self-penalize". He wants to penalize according to how difficult the estimation procedure is. This is exactly the principle of parsimony. – user10525 Apr 22 '12 at 13:22