Overfitting a regression model

What are the effects of over-specificating a linear regression model?

Could it be said that overfitting is less important when the goal is to only estimate the relationship between dependent and independent variables, and not making any predictions for the observations outside the given data set?

• I don't believe so. Suppose you include one predictor $x$ that has a real effect on $y$, but also a bunch of others that are proxies for $x$. The true effect might be masked due to overfitting and your estimate would be poor. – dsaxton Mar 18 '16 at 15:43
• @dsaxton In other words, you are saying that coefficients would be biased? – Quirik Mar 18 '16 at 15:47
• Yes, pretty much. – dsaxton Mar 18 '16 at 16:00
• I agree with @dsaxton, but I wouldn't call the result a bias. I would say that the resulting estimates would have such a high variance to be essentially untrustable and useless. Generally (but not always) bias results from undercutting, large variance from overfitting. – Matthew Drury Mar 18 '16 at 16:50
• @MatthewDrury Right, I was reluctant to attribute it just to variance because that seems to suggest it could be cured with a large enough sample size, but I'm not sure if that's the case in these situations. – dsaxton Mar 18 '16 at 18:42