0
$\begingroup$

enter image description here The question is

If you are forced to choose between red and green (they have the same training error) which one would you choose?

And I have the answer as below,

Green! Beacuse The slope is smaller, so predictions are less sensitive to having ‘w’ exactly right.

Though, I can not understand why do we want the smaller slope!

I would appreciate if somebody explain it.

$\endgroup$
0
$\begingroup$

I wouldn't advise choosing neither, but if you're forced to choose one, the essence of small slope is lower prediction variance. That's why, in the explanation, it says "predictions are less sensitive". High variance, meaning high uncertainty, is typically not welcome, since you don't want your predictions vary abruptly.

For example, suppose you need to choose one of the following estimations for your given data: $Y_1=2X+1$, $Y_2=0.5X+1$, having the same MSE. Then, variances of these predictions are $\text{var}(Y_1)=4\text{var}(X)$, and $\text{var}(Y_2)=0.25\text{var}(X)$. The latter has substantially lower variance compared to the former.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.