# Do classical statistical methods assume that they contain or have access to the true distribution?

I was reading recently about Minimum Description Length (MDL) that does not have to assume that the true model is in the model class. When this feature of MDL is highlighted in the literature, the standard statistical inference is usually criticised at the same time that its methods are designed under the assumption that they contain the true distribution. The pioneering papers of Rissanen and others mention this.

My question is: Which of the classic statistical methods make the assumption that they have the true model in the model class or that the true model has to be considered in the analysis? What happens to the results if the true model is not in the model class (which will be the case most of the time, I believe)? Are the results still valid in such cases?

I can clearly see why MDL does not assume that it has access to the true model, but it is not clear to me why the opposite is true in more traditional statistical models.

• "...the assumption that they have the true model in the model class..."---this is assumed in all of classical (parametric) statistics. E.g. take any foundational result---just to name a few, Neyman-Pearson, existence of most powerful or minimax test, Maximum Likelihood Principle, etc. Some results, however, can be extended to the setting where the true model does not lie in the class of models being considered, i.e. under mis-specification. For example, the maximum likelihood estimator minimizes KL divergence between the class over which likelihood is maximized and the true model. – Michael Sep 4 '20 at 21:49
• MDL drops the whole concept of "true model". – carlo Sep 5 '20 at 1:22

Perhaps an example will assist here. Suppose you have a parametric model for data $$\mathbf{x}$$ that depends on an unknown parameter $$\theta \in \Theta$$. One of the concepts formed in classical statistics is the idea of "consistency" of an estimator. Broadly speaking, for an estimator $$\hat{\theta}$$ of the parameter $$\theta$$, this means that $$\hat{\theta} \rightarrow \theta$$ (in some appropriate probabilistic sense) as $$n \rightarrow \infty$$. In order for the estimator to be considered "consistent", this property must hold for all $$\theta \in \Theta$$. Thus, so long as the assumed model form for the likelihood function is correct, a consistent estimator will tend to estimate well in the long run --- this is guaranteed regardless of the unknown parameter value. (Note here that the theoretical derivation of the consistency property involves looking at the behaviour of an estimator under an assumed true parameter value, and varying the assumed true value over the entire parameter space. However, once a particular estimator is known to be consistent within a particular model, for applied work we do not assume that the true parameter is accessible.)