# What is a 'true' model?

A short question, but I am somehow unable to find any concrete answer. I suppose it means that the model is as good as it can be? Containing all relevant variables and hence not suffering from any bias such as omitted variable bias? I am specifically referring to a linear regression model.

• Imagine that you perform a physics experiment relating pressure and wavelength. The physical relationship between these to properties is not yet known, so you gather experimental data for analysis. In my example the physical relationship is linear, but the gathering of data has noise that obscures this - so that you take repeated measurements before analysis. After regression, the fitted parameters are determined with some level of statistical certainty withing whose bounds the "true" values should occur. – James Phillips Mar 17 '19 at 21:58
• Alright, but what if I am given a certain 'true' model which I have to estimate. What does that mean? – Anon Mar 17 '19 at 22:01
• Your estimate will have some bounds on the statistical certainty - or statistical uncertainty - of the estimated values. – James Phillips Mar 17 '19 at 22:07
• Does it have any implications for omitted variable bias? – Anon Mar 17 '19 at 22:34
• All models are false. Some models are more useful than others. Also: the map is not the landscape. – Alexis Mar 18 '19 at 1:02

No, the true model is the data-generating model/process, which is only known ex-ante if you assume the underlying model beforehand (e.g. simulations or theoretical models). If you only observe data, you do not know what the true model is. You try to find a model that explains data the best, which does not mean that it is the true model.

In fact, it is possible that you find a model that "fits" just as good as the true model (if you would have known), even though true model and assumed model are different. This happens, for instance, when you have hidden variables that you do not know of that you never see. Drawing inference from these is almost impossible.

In a regression context you have variables $$(y_i,\mathbf{x}_i)$$ and you are seeking to describe the behaviour of the first element conditional on the second element. The model posits a class of possible conditional distributions of $$y_i$$ given $$\mathbf{x}_i$$, and the true model is the true conditional distribution. In my view, it is best to avoid equating this to the "data generating process" since that is an additional causal hypothesis, and it brings in a large number of strong assertions that are impossible to prove (e.g., that probability is an embedded metaphysical property of nature, and not just an epistemological tool for reasoning).

Suppose you accept the view that the "true model" is a synonym for the true conditional distribution. It is still nice to be able to give an operational meaning to this (i.e., a meaning framed in terms of observable data), if possible. To do this, suppose you are willing to assume that you have a potentially infinite set of observable data, manifesting in an infinite sequence $$\mathscr{R} \equiv \{ (y_i,\mathbf{x}_i) : i \in \mathbb{N} \}$$. (In a given problem you will only observe a finite amount of data, but our assumption is that there is no finite limit to the amount of data we could collect in theory.) Define the limiting empirical distribution function $$F_\infty: \mathbb{R}^{m+1} \rightarrow [0,1]$$ by:

$$F_\infty(y,\mathbf{x}) \equiv \lim_{n \rightarrow \infty} \frac{1}{n} \sum_{i=1}^n \mathbb{I}(y_i \leqslant y, \mathbf{x}_i \leqslant \mathbf{x}) \quad \quad \quad \text{for all } y \in \mathbb{R} \text{ and } \mathbf{x} \in \mathbb{R}^m.$$

If the sequence $$\mathscr{R}$$ is exchangeable then it follows from the strong law of large numbers that $$F_\infty$$ is almost surely equal to the true distribution $$F$$ (i.e., we have $$\mathbb{P}(F_\infty = F)=1$$). This means that the conditional distribution induces from the limiting empirical distribution of the sequence is the true conditional distribution of $$y_i$$ given $$\mathbf{x}_i$$ --- this gives an operational meaning to the "true model".

• I have a point about your argument. Shortly, in economic/econometrics context what meaning are things as: true joint/conditional distribution? True partial/total correlation and/or true regression? What mean "not true"? See here stats.stackexchange.com/questions/399671/… – markowitz May 7 '19 at 7:34
• It seems me that correlation/dependence is a "free" concept, beyond it we have only causation. – markowitz May 7 '19 at 7:41

You're right. It's very hard to find a good discussion of this. My thoughts: The "true" model is not a model of how the data were actually generated but rather a hypothetical "generating model" that generates data with the distribution P(Y|X), where X are the independent variables in your statistical model, and satisfies Gauss-Markov (see Wikipedia), so error (not residuals!) are I.I.D. and mean zero. Omitted variables are irrelevant to these conditions. Literally an infinite number of generating models (with different combinations of causal factors) can generate data with the same P(Y|X). Omitted variable bias is simply not relevant to statistical modeling the way it is described in statistical textbooks. Some of this is in Gelman and Hill. Another good source is Shalizi's draft for a textbook (all googleable). See my comment below for the most comprehensive source I've found that offer's an answer to this question.

• some other sources: the first few pages of Pearl's "Trygve Haavelmo and the Emergence of Causal Calculus". Morgan and Winship "Counterfactuals and Causal Inference. Methods and Principles for Social Research" have a short discussion of a "true" model defined in statistics textbooks and a real, generating model. – JWalker Mar 17 '19 at 23:27
• it seems me that your position is not consistent, see below. Agree? – markowitz May 7 '19 at 7:28
• markowitz - see Ben's more formal response, which is consistent with my response. A better source, and the most comprehensive source I've found, is: Spanos, A., 2006. Where do statistical models come from? Revisiting the problem of specification. In Optimality (pp. 98-119). Institute of Mathematical Statistics. projecteuclid.org/euclid.lnms/1196283957 – JWalker May 8 '19 at 10:40
• First of all thanks for the references you gave. However the contradiction that I have evocated is explicated below (my answer), it come from Pearl paper citation and it is not solved by the others paper cited. The problem is not trivial but at least terminologically we have to says that Data Generating Process concept have a structural substantive meaning see Spanos (pag 100) that you have cited. – markowitz May 8 '19 at 15:17
• Then I remain also stronger in my position because Morgan and Winship (pag 161/165; you cited) speak about “true model” as structural/causal. This mean that "true model" in econometrics, and then any social science, have substantive meaning; this was my argument (see below). I think that RJAL speak about econometrics; omitted variables problem is typical in it and come exactly from true model paradigm. – markowitz May 8 '19 at 15:17

It seems me that the the position of Gkhan Cebs is correct, true model and data generating process/model are synonym.

The position of JWalker is strange because it sustain that meaning of true model stay only in joint probability distribution but this position is precisely contradicted in the Pearl's paper he cited "Trygve Haavelmo and the Emergence of Causal Calculus". Honestly Pearl never speak about "true model" and only of "data generating mechanism" but JWalker cited the paper as referee for true model meaning. The reason can be only that he consider true model and data generating process as synonym, and it seems me correct but this fact posed the JWalker answer in contradiction.

However JWalker and RJAL have right when says that the "true model" meaning is very difficult to find and then to understand. In econometrics textbooks the meaning of "true model" is skipped and/or unclear. Sometimes is said that it has theoretical/causal meaning, sometimes only statistical one, sometimes else nothing is said. It seems almost a mystery. This fact produce great confusions.

Maybe in some statistical text something like the "true model" can be used without structural meaning. However I think that the correct interpretation for the true model in econometrics is like: structural linear causal equation. Like here: linear causal model

These discussions are strongly related:

Regression and causality in econometrics

In regression analysis what's the difference between data-generation process and model?