Skip to main content
1 of 2
civilstat
  • 4.6k
  • 21
  • 37

You wrote:

With one feature we will have a straight line in a plain, and by adding one feature we will have a straight plane in a 3 dimensional space.

Imagine that your response variable, $Y$, truly depends on two features, $X_1$ and $X_2$. Let's even say that a linear model is true:

$Y = f(X_1, X_2) + \mathrm{noise} = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \mathrm{noise}$

If these betas are nonzero, then in the 3D space, $f(X_1, X_2)$ is a tilted plane. It's not curved (no polynomials, interaction terms, etc.). But it sits at an angle to each of the $X$ axes.

Now, let's say that you start by ignoring $X_2$ and just trying to fit the model

$Y = \beta_0 + \beta_1 X_1 + \mathrm{noise}$

Then you will also get a plane as you pointed out. This plane is tilted with respect to the $X_1$ axis, but not with respect to the $X_2$ axis. This is a biased model. Since you don't include $X_2$ in your model, there's no way for the fitted plane $Y = \hat\beta_0 + \hat\beta_1 X_1$ to match the true plane $Y = \beta_0 + \beta_1 X_1 + \beta_2 X_2$, not even on average across different samples from the population.

civilstat
  • 4.6k
  • 21
  • 37