2
$\begingroup$

I am new to machine learning and I just used SVM for the first time to analyze my dataset... Now I have created a figure that displays the training and testing error of the model as a function of variable test/train set size.. I need to say something useful about this graph but I do not understand how the training error is defined and how this correlated to the testing error.. Can somebody clearify this for me? (And is it right this thing is called a learning curve?)

graph

$\endgroup$
1
$\begingroup$

A learning curve is a plot of the training and cross-validation (test, in your case) error as a function of the number of training points. not the share of data points used for training. So it show how train/test errors evolve as the total data set increases. See here for examples and more detail.

The 'train error' would be the error (according to your loss function) achieved for the training set, and the 'test error' means the same for the test set. See here for more detail.

If I interpret your chart correctly, then the fraction of data you are using to test your model increases to 90%, while the error decreases for the 'test' data, while it increases for the (simultaneously shrinking) train set.

In other words, as you are training your SVM model using fewer and fewer data, the 'train error' increases, which makes sense. It is a bit odd that the test error would be decreasing as you reduce the size of the train set, so perhaps I am misinterpreting your chart?

$\endgroup$
  • $\begingroup$ Thank you a lot Stefan! I think you are right. It is odd that the test error line decreases upon increasing the test data size.. Is it possible that the model is less well trained on less training data and therefore has poor predicting quality? Or is it not true that the test error is defined as the ability to predict the label of test data? $\endgroup$ – Afke Jun 22 '16 at 15:34
  • $\begingroup$ If the increasing share for 'testing' means that you are reducing 'training' at the same time, then one would expect the test error to increase because it does indeed measure the error in predicting the test label. That error should of course increase rather than decrease for a poorly trained model. $\endgroup$ – Stefan Jun 22 '16 at 15:41
  • $\begingroup$ Is it possible that at 0.9 overfitting occurs? Since the training error is low and the test error is enormous? $\endgroup$ – Afke Jun 22 '16 at 15:48
  • $\begingroup$ Exactly, overfitting means that the training error becomes very low as the model fits 'too well' on the know, data, but fits poorly on unknown data. $\endgroup$ – Stefan Jun 22 '16 at 15:51
  • 1
    $\begingroup$ @zthomas.nc - see updated link in answer. $\endgroup$ – Stefan Jan 30 '17 at 0:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.