# Can a Linear Regression Model (with no higher order coefficients) over-fit?

How can a straight line (or plane) over-fit? My question is not about polynomial regression (although it too is considered 'linear'), but regarding a linear regression model with no higher order features, such as the following equation:

$$y = \theta_0 + \theta_1 X_1 + \theta_2 X_2 + \theta_3 X_3 + \theta_4 X_4 + ...$$

I did go through other answered questions on this web which speak of poor generalization of a linear regression model with too many features (and that is in fact over fitting), but geometrically speaking, I cannot understand how can a linear model over-fit.

Here is Prof Andrew Ng's example of over-fitting shown geometrically. As far as I can see, a linear model (with no higher order features) can only under fit (the first figure depicting logistic regression):

Similar question: Overfitting a logistic regression model

Assuming the real model is: $$y_i = \beta_0 + \beta_1 X_{i1} + \epsilon_i$$ but you add a factor $X_{2i}$ which not relate the $y_i$ to model and fit the new model $$y_i = \beta_0 + \beta_1 X_{i1} + \beta_2 X_{i2} + \epsilon_i$$ In general, you will get a $\hat{\beta_2} \neq 0$, then if you run the model to predict something including factor $X_{i2}$, you will suffer over-fit.
• What is $\hat{\beta}_2$? – Viktor Glombik Sep 1 '19 at 21:07