4
$\begingroup$

Just a note concerning the splitting and using of the dataset. So if I understand this correctly, we use the validation set to confirm that parameter changes made are giving us positive results. And the testing set is put aside until the very end after we have optimized the model.

So if I had a default SVM, I would use the validation set after say changing the kernel or the gamma to see the performance difference. Then once I have my final optimized model I would use the test set to see how well it generalizes to unseen data?

Improve this question
$\endgroup$
0

1 Answer 1

Reset to default
3
$\begingroup$

In simple words, yes. What you have written makes perfect sense. It is advised to split your data into 3 parts :

  1. Training set- This is the actual set where you train your model to get parameters.
  2. Cross Validation set- Cross validation sets are used to indicate that your model has actually learned something useful from the training data.
  3. Test set- It is used to calculate the accuracy of your model.

Mostly, you get only the training data. So you can split the dataset into training, cross-validation, and test set. The ratio to which is

train:cross-validate:test=60:20:20

This is what I usually choose. This is advisable to do because if you only use one set for training and testing, there's uncertainty about how the model is going to perform in the real world with unseen data.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ The train/dev/test split depends on a problem and there is not single "best" or "universal" split. There are cases where you'd use 50% or 1% of the data for testing. $\endgroup$
    – Tim ♦
    Nov 2, 2018 at 8:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.