I have a data sets and want now find a model to predict wages.

I read that just cutting the data sets into 2 parts by a percentage number to get the training and test data set is not efficient. Instead I should use cross-validation to generate test and trainings data sets and then compute difference models with different lambdas and choose then right lambda via cross validation.

I don't understand why its not efficient for example to just set 80% of the the data as a training and the rest 20% as a test data set?

Is what I am talking about above correct and what is the advantage of this procedure?

  • $\begingroup$ Can you provide a source or context for where you heard this? It would help to know the context in order to know why these statements were made. $\endgroup$ – enumaris Jun 21 '18 at 22:19

You can certainly use an 80/20 (or a different proportion) split, especially if you have a lot of data. However, there is a bit of risk involved in this method: your results will depend heavily on how your data are divided. If you have a relatively small dataset, and your test set ends up differing meaningfully from your training set, the estimate of out-of-sample error will be poor.

Using k-folds cross validation lets us get some more use out of each data point. Let's consider five-fold cross validation as a simple example. We divide the data randomly into five subsets. We train the data on four of the subsets and test on the fifth, repeating until each of the five subsets has served as our test set. We evaluate the model by averaging the results obtained from each of the five training/test runs.

You can think of this as being a "more efficient" method in terms of use of the data. Each data point is used in the test set once, and each data point is included in the training sets four times. By using a simple 80/20 split, we may not be getting as much information as we can out of our data: only a relatively-small subset of points are ever included in the test set.

However, k-fold cross validation is not computationally efficient compared to using a simple split of the data into training and test sets. You'll need to train your model k separate times, which can be demanding if k is large and/or the model fitting is computationally intensive.

| cite | improve this answer | |
  • $\begingroup$ Thank you for your answern. I allready read about cross validations and it makes sense to me. BUt i stillt dont understand why people divide theire data in the first step. Then use cross validation on training data to get the best Modell. And then at the end they apply the model on the test data set to see how his predictions accuary is. Why dont just to cross validation on the whole data set`? $\endgroup$ – Dima Ku Jun 23 '18 at 10:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.