0
$\begingroup$

What is the difference between Gradient Descent and Ridge regression?

We use ridge regression for overfitting problem when the Mean Squared Error for test dataset is high. I think that we can use gradient descent instead of ridge regression by using it on test dataset. This way we can the slope and intercept which has has the least MSE. Thus we can get the best fit line like this.

Please help me to understand the difference between Ridge regression and Gradient descent for linear regression.

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ Gradient descent is the way you optimise your loss, it's not a type of regression or end-to-end ml algorithm. $\endgroup$
    – gunes
    Commented Jan 14, 2021 at 11:16
  • $\begingroup$ We don’t optimize a loss function on a test set, so what you’re proposing is confusing. Perhaps you can give an example. $\endgroup$
    – Dave
    Commented Jan 14, 2021 at 12:29

1 Answer 1

Reset to default
1
$\begingroup$

The parameters in a ridge regression can be estimated using gradient descent. Gunes basically answered this above - but gradient descent is a way of estimating the parameters of a model. It's like ordinary least squares - except a lot more versatile and can be applied to a lot of different optimisation problems.

Improve this answer
$\endgroup$
1
  • $\begingroup$ In other words, gradient descent is an optimization algorithm because it discovers a parameter configuration which is optimal. Ridge regression is a model, because it describes a mathematical relationship between the features and the outcome. $\endgroup$
    – Sycorax ♦
    Commented Jan 14, 2021 at 18:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.