Taking an OLS model (actually, is it a "model" or an "estimator"?) as an example, there are several assumptions (such as strict exogeneity and spherical errors) which are important for the consistency of the estimate and efficiency of the estimator.

It is clear that if we are interested in making inference about the influence that different predictors have on the outcome, the consistency property is crucial. However, if we are only interested in making predictions by applying the estimated linear relationship to a different set of observations (where we don't expect significant differences between samples, e.g. approximately the same range), do we really care that much about consistency?

Add 1

I did some simulations comparing out-of-sample MSE of OLS, maximum likelihood (MLE), and method of moments (MM) (i.e. using sample moments) for a model given by

$$y_i = x_i \exp(\sigma z_i)$$

with $\sigma = 0.5$ and equally-spaced $x \in [1,10]$ for $n \in \{2^k, k=4,...12\}$. The expected value of $y$ is linear in $x$

$$E[y|x] = x\exp(0.5 \sigma^2)$$

but the errors are not exogenous

$$E[\epsilon | x] = E[a + bx - y| x] = E[a + bx - x\exp(\sigma z)| x] = a+bx-x\exp(0.5 \sigma^2) \ne 0$$

and not spherical

$$Var[\epsilon | x] = Var[a + bx - y| x] = Var[y| x] = x^2 Var[\exp(\sigma z)| x] = x^2 (e^{\sigma^2}-1) e^{\sigma^2}$$

For MLE and MM, the correct log-normal model was specified. The results are shown in the plot below which shows mean MSE. For this set up, the OLS underperforms for samples less than 128, but the standard deviation of MSE is around 10% smaller for $n \le 32$.

enter image description here

  • 1
    $\begingroup$ In my experience, no. As you've noted, the assumptions for OLS are only there to make sure the estimation is unbiased, consistent, etc etc. If all you care about is prediction error, then there is no need for homogeneity of variance, for instance. It makes evaluating the predictive quality of your model a little more nuanced, but no one would knock you for it. $\endgroup$ Feb 6, 2019 at 19:24
  • $\begingroup$ @DemetriPananos Thank you for your comment. Would you happen to know if it can be demonstrated / proved that the prediction error (say, measured with the MSE) is indeed not affected by the standard assumptions not being satisfied? Thanks $\endgroup$
    – Confounded
    Feb 6, 2019 at 19:29
  • $\begingroup$ I don't know of any proofs, but the point is that in prediction, we don't make any assumptions about the MSE prior to commencing modelling, so there is no way in which the assumptions could affect it. $\endgroup$ Feb 6, 2019 at 19:35
  • 1
    $\begingroup$ OLS is an estimator, not a model. Regarding consistency, there is the classical notion and the predictive one; see the thread T-consistency vs. P-consistency. $\endgroup$ Dec 20, 2019 at 16:40


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.