0
$\begingroup$

This question already has an answer here:

I am a beginner in machine learning. I have a question why adding more samples to a data set is equal to adding regularization term to the ordinary least squares loss function? (In other words why can I add more samples to my data set and solve OLS instead of solving ridge regression?)

$\endgroup$

marked as duplicate by whuber regression Oct 2 '18 at 13:45

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

0
$\begingroup$

Edited:

There is an equality: check out @whuber's answer.

Originally, I understood your statement as: "Adding/Collecting more data (i.e. real data) is equivalent to ridge regression". And, in the following answer, I tried to address this. I'm leaving it for anyone who has confused the question like me.

Their service to regression mechanism resembles each other. If you have little data, it is easier to overfit it. Adding more data can increase your generalization performance directly. Regularization in its purest form tries to achieve this same purpose.

$\endgroup$
  • $\begingroup$ Contrary to your initial assertion, my answer in the duplicate thread explicitly shows there is a mathematical equivalence. I don't believe we have a difference of opinion but only a difference in how we understand the current question. $\endgroup$ – whuber Oct 2 '18 at 13:46
  • $\begingroup$ I believe there is a mathematically provable because it is my ML class question. $\endgroup$ – soroush Oct 2 '18 at 13:53
  • 1
    $\begingroup$ Oh, right! I got the question wrong way. Ridge regression can be written as if we add data; but I thought of it like collection of new/real data for your model is equivalent to ridge regression. Should I delete this or leave it?? $\endgroup$ – gunes Oct 2 '18 at 13:57
  • 1
    $\begingroup$ Your answer is not completely wrong, you can edit it. Actually, I am looking for mathematical proof and that exact data to be added. $\endgroup$ – soroush Oct 2 '18 at 14:08
  • 1
    $\begingroup$ A good way to prevent such confusion is to state--usually at the outset of your response--how you understand the question. That habit has spared me a lot of grief over the years :-). $\endgroup$ – whuber Oct 2 '18 at 14:18

Not the answer you're looking for? Browse other questions tagged or ask your own question.