# When will a less true model predict better than a truer model?

In "To Explain or to Predict?", Pr. Galit Shmueli said that sometimes a less true model can predict better than a truer model. Why is it so? When will it happen? How does it happen? Is explanation a trade-off for prediction in models?

• Have you read Shmueli's article? From reading the abstract, it seems that his article is aimed at precisely answering these questions. Are there particular arguments he makes that you have questions about? – Peter Ellis Feb 10 '12 at 5:32
• Note that the full text is available via Google Scholar: scholar.google.com/… – Jack Tanner Feb 10 '12 at 5:55

I will try to give a quick sense of the idea. Let me first define some terms. To start with, what Shmueli means by "true" model is the actual data generating process; the closer your estimated model is to the real data generating process, the truer it is. For instance, if $\beta_1=.5$, and one model fit yields $\hat{\beta}_1=.6$, that's truer than another fit that yields $\hat{\beta}_1=.7$. On the other hand, predicting better means getting your $\hat{y}$'s as close as possible to the actual $y$'s, especially for out-of-sample data. Notice the differences in goals here (because that's crucial to understanding the issue): getting $\hat\beta$'s as close as possible vs. getting $\hat{y}$'s as close as possible. So Shmueli's point is that sometimes your $\hat{y}$'s can be closer to the actual $y$'s when your $\hat\beta$'s are estimated by a process that, on average, yields values a little further from the true $\beta$'s. Now, how is that possible?
The key is that there is variance associated with parameters estimated from sample data. For a given sample, sometimes the maximum likelihood estimate happens to be further from the true value and sometimes closer. It is quite possible to have a situation where the variance of the sampling distribution of a parameter estimate is so large that $\hat\beta$'s routinely bounce so far out around their true value that they are not worth much. The thing to remember here is that classical statistics are based on what are called the 'best linear unbiased estimators', that is, the estimators that have the lowest variance of all the unbiased estimators. However, there can be other ways of attempting to get an estimate that are not unbiased. Typically, these have been developed within machine learning (a subfield of computer science). It is possible in some cases to have an estimate that doesn't tend to bounce as far from the true value, even though the sampling distribution of that estimate is not centered on the true value (i.e., it is biased). Given all of this, what matters for the accuracy of your predictions is how the inaccuracy due to the induced bias trades off vis-a-vis the inaccuracy induced by the high variance of the BLUE parameter estimate (hence the name). Specifically, if the inaccuracy due to higher variance is greater than the inaccuracy due to the bias, the less true model will give the better predictions.