# why ridge regression only decreases slope and not increases it?

I was following the below example from 'StatQuest with Josh Starmer' youtube channel.

The example is pretty simple: red line is the usual 'least squares' (for the red points), and the blue one is ridge regression line (for the red points); where we sacrifice a bit of error in the test data, but it will fit better all data (green +red dots).

I do understand the above, and it makes sense; but what if the all the real data ends up being above the line? Why ridge regression only assumes that all the remaining data could be better fit with a smaller slope and not a larger slope?

• – Sycorax
Apr 13, 2019 at 19:35
• Looks like a mathematical explanation/proof. I understand how ridge works and we are minimizing it, we will never get a greater slope than 'least squares'. My question is why? per my view there is a 50/50 chance that the data points will be either above or below, why we only search 'below'? Apr 13, 2019 at 19:42
• I agree, but, any penalty selected will only make my slope lower (or stay the same), but never larger! Why we assume that a lower slope might fit better? why don't we assume that a larger slope might fit it better? Apr 13, 2019 at 21:46
• Nobody assumes that: it's a result of the model. In fact, when you introduce more than one explanatory variable, the sizes of the slopes actually can increase with Ridge Regression.
– whuber
Apr 13, 2019 at 22:41
• First, @whuber 's comment seems like it's close to an answer, that answer being "it doesn't necessarily do that". Second, I'm not sure your understanding of ridge regression is correct. Ridge allows for some bias in the parameter estimates in order to reduce the variance of those estimates. Apr 14, 2019 at 12:24