I can not see any difference between Ridge Regression and Linear Regression

MY understanding, The point of ridge Regression is based on the training data we find the best line that fits training data.

Best line means minimum RMSE

then try to play with the line sloop to get better results through n-fold cross validatin!.

isn't easier and simpler to use all dataset (both training and test) to build this line and find sloop through

$y\ =\ \beta_0+{\beta_1x}_1$

$\beta_1\ =\ \rho\frac{\sigma_y}{\sigma_x}$

$\beta_0\ =\ \mu_y\ -\ \mu_x\beta_1\ $

$\rho\ =\ [(x-μx)(y-μy)] [(x-μx)2][(y-μy)2]$

$\sigma_x=\ \sqrt{\frac{{\sum{(x-\mu_x)}}^2}{n}}$

$\sigma_y=\ \sqrt{\frac{{\sum{(y-\mu_y)}}^2}{n}}$

the linear regression will give us the best fit.

if i misunderstood.

please tell me what is the difference between these models before down rating my question.


  • 1
    $\begingroup$ Contemplate what "best" means, because different forms of regression often differ in how they measure how "good" the fit is. $\endgroup$
    – whuber
    Jun 27, 2019 at 12:07
  • 1
    $\begingroup$ Then please review the definition of Ridge Regression, because it explicitly does not minimize the RMSE. $\endgroup$
    – whuber
    Jun 27, 2019 at 12:13
  • 1
    $\begingroup$ Ridge regression is not just similar to linear regression... instead, it is exactly like linear regression. It is a specific type of linear regression, where the 'linear' refers to the model $y = \beta X$. How ridge regression differs from the most common type of linear regression, ordinary least squares regressions, is in the added penalty that makes one favor solutions with small effect sizes (this is advantageous when you have lots of regressors for which you can reasonably expect that most of the associated effects should be equal to zero or close to it). $\endgroup$ Jun 27, 2019 at 12:18
  • 1
    $\begingroup$ Note the double negation. I am not saying that the other line is not y=ΣBX. The other line will still be like y=ΣBX but with other coefficients B. It depends on what you consider 'best' line. If your goal is to minimize the least squares residuals than there is only a single unique solution. But linear regression is not synonymous/equal to minimizing least squares residuals. There are other ways to fit a line that do not minimize least squares residuals, but instead minimize something else. $\endgroup$ Jun 27, 2019 at 12:40
  • 1
    $\begingroup$ @MartijnWeterings & whuber thanks heaps. things are much clearer now. really appreciate your help. I will re-read the topic i think I will understand it better now. $\endgroup$
    – asmgx
    Jun 27, 2019 at 12:48


Browse other questions tagged or ask your own question.