I encountered a question while learning:
While doing a homework assignment, you fit a Linear Model to your data set. You are thinking about changing the Linear Model to a Quadratic one.
- Using the Quadratic Model will decrease your Irreducible Error
- (Correct answer) Using the Quadratic Model will decrease the Bias of your model
- Using the Quadratic Model will decrease the Variance of your model
- Using the Quadratic Model will decrease your Reducible Error
And an explanation for it:
Introducing the quadratic term will make your model more complicated. More complicated models typically have lower bias at the cost of higher variance. This has an unclear effect on Reducible Error (could go up or down) and no effect on Irreducible Error.
I wonder what is the difference between reducing bias and reducing reducible error? I thought one always implies another.