I am taking a course on machine learning and in one problem I should perform a Ridge regression to fit some given data to a known model. I was wondering if, in this case, there are any advantage in splitting the data into training set and test set. At the end of the day I am performing Ridge regression on a known model, hence there is no risk of overfitting. Is there any other reason to split the data in such way?

  • 3
    $\begingroup$ Why do you say there is no risk of overfitting? $\endgroup$ – Dave Feb 5 at 15:59
  • $\begingroup$ Maybe I am wrong, but I assume it since I know the model that generates the data. $\endgroup$ – laklica Feb 5 at 16:09
  • 2
    $\begingroup$ How does your knowledge of the data generating process enter into the ridge regression? Ridge regression alone can also overfit. $\endgroup$ – Stephan Kolassa Feb 5 at 16:18
  • $\begingroup$ Because it was given in the statement of the problem. $\endgroup$ – laklica Feb 5 at 17:01
  • 1
    $\begingroup$ It's not clear what exactly is meant by "known model". Known to whom? Why does it matter? $\endgroup$ – Andris Birkmanis Feb 5 at 18:45

Your evaluation should always be performed on a holdout set. In the simplest setups, this reduces to train & test sets. So, you should split. Ridge regression has regularisation mechanics but it may not save you from overfitting.


Not the answer you're looking for? Browse other questions tagged or ask your own question.