0
$\begingroup$

Could someone explain (in simple way) what does mean of validating model ?
I tried to understand it, but I didn't managed to.

  1. I can do cross-validation, but I am not sure about if it is validation.
  2. What is the aim of validation ?
  3. What are ways of validation ?

As I mentioned above - I don't expect very long expactation (that I can find in book), I have a problem with intuition here.

$\endgroup$
0
$\begingroup$

Usually, when you train your model, you split your dataset into two:

Train set

Train set is there for your model, so it can learn from it.

Test set

After your model is trained, you will show it data it has not seen before and see how well it will behave.

And now comes the validation part.

You are interested in how your model behaves in general. How well it can predict your target values. Is it poor estimation? Is it good one? And what if your model just learnt the pattern in training set (=overfitting).

So you must do validation on training set:

  • What is the learning error?
  • You do cross validation to see how it really behaves and that it does not overfit (check this question)

And you must do validation on test set:

  • To be sure that your model just did not learn the train set because you want to catch the big picture, not copy your train set

When validating, you can use various measures depending on your situation:

Accuracy, Precision, Recall, ROC curve, AUC, F1, etc.

$\endgroup$
  • $\begingroup$ Tell me please, cross validation is method of validating ? $\endgroup$ – user113135 Apr 22 '16 at 11:57
  • 1
    $\begingroup$ Yes, exactly. It tells you how the model behave on random subsets and if done correctly, there is high chance that your model learnt the pattern in your data and did not overfit. For more info about CV, have a look at the link I provided. $\endgroup$ – HonzaB Apr 22 '16 at 13:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy