0
$\begingroup$

It is clear to me that adding polynomial features fixes high bias. So far so good. But there is also the claim that adding more (non polynomial) features fixes high bias. I don't see why. In my humble opinion it will not fix it since linear regression will stay linear regression if a new, non-polynomial feature is added. With one feature we will have a straight line in a plain, and by adding one feature we will have a straight plane in a 3 dimensional space.

$\endgroup$
1
  • $\begingroup$ Consider changing the topic of your question to "Why adding polynomial features..." instead of "additional". $\endgroup$ – user209249 Apr 14 '20 at 21:33
0
$\begingroup$

If you think about it, adding polynomial features is somewhat equivalent to polynomial regression. Instead of adding a polynomial term to the model, you add it to the features. You will end up with the same equations. A polynomial model has less bias because it can fit more complex functions.

Linear regression with two components: $$ LinReg(x_1, x_2) = a_0 + a_1 x_1 + a_2 x_2$$

Polynomial regressions with one component: $$ PolyReg(x_1) = a_0 + a_1 x_1 + a_2 x_1^2$$

If you set $x_2 = x_1^2$ you end up with the same expression.

$\endgroup$
1
  • $\begingroup$ Sören: You misinterpreted my question. I understand why adding polynomial features fix high bias. I don't undertstand why non-polynomial features do so. $\endgroup$ – EL Dendo Apr 14 '20 at 21:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.