0
$\begingroup$

I've been reading up on machine leaning and keep seeing data sets split up into a training, test, & validation set. Here's what I think the differences are based on what I've read:

training set => choosing the features that you think are most important in predicting your label

test set => splitting your data into training & testing sets (e.g. 75% of your features used to predict labels)

validation set => new, real world data never been seen before

Are these distinctions accurate?

$\endgroup$
2

1 Answer 1

1
$\begingroup$

Basically you do a train-test split on your data when you're doing a machine learning task. The training set is the data that your algorithm will learn from. For supervised learning, you usually include the ground truths in when feeding. This is the part where you feed it to the algorithm. While the test set is to test whether or not the algorithm was able to learn what you wanted to teach it. For testing supervised machine learning algorithms, the ground truths aren't included and your algorithm would have to make predictions already. The usual train-test split can be either 70-30 or 80-20. It really depends on you. The validation set or holdout is the data that your algorithm that hasn't seen before either in training or testing. Basically this would be like a test set. However, the difference is that it somehow validates the accuracy of the training set if it's close enough else, if it's not, the model might be overfitted.

$\endgroup$
3
  • $\begingroup$ by ground truths you mean what you're trying to predict ? $\endgroup$
    – e1v1s
    Oct 26, 2017 at 3:07
  • $\begingroup$ Yes. It's basically the true value of something. $\endgroup$
    – Jessie
    Oct 26, 2017 at 3:18
  • $\begingroup$ I would rearrange this: I think the usual paradigm is to have a train-test split before you do anything else, putting the test data to one side until right at the end. You then use the training set for model selection and hyperparameter tuning by choosing a validation set from within the training data (possibly multiple times with cross-validation) and with a chosen model and hyperparameters then train your final model. As the ultimate step, you predict for the test data using this final model to see how it performs on previously unseen data. $\endgroup$
    – Henry
    Apr 4, 2023 at 21:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.