I get output like this ..

=== Stratified cross-validation ===
=== Summary ===

Correctly Classified Instances          85               53.125  %
Incorrectly Classified Instances        75               46.875  %
Kappa statistic                          0.0625
Mean absolute error                      0.4688
Root mean squared error                  0.6847
Relative absolute error                 93.75   %
Root relative squared error            136.9306 %
Coverage of cases (0.95 level)          53.125  %
Mean rel. region size (0.95 level)      50      %
Total Number of Instances              160     

What is the generalization error here?

  • 2
    $\begingroup$ To make this a statistical question (instead of a software-specific question which fall under the category of "too localized" question), you probably want to broaden the scope of your question and ask how to compute generalization error for a particular dataset and algorithm/classifier. $\endgroup$ – chl Nov 21 '12 at 21:14
  • $\begingroup$ @chl Thanks for formatting .. I have been trying to search on net but specifically given a confusion matrix or the output of this kind how can I get the error ... I was also confused about the different errors provided by weka ... Sorry if I sounding stupid, just started with this thing .. $\endgroup$ – Fox-Kid Nov 21 '12 at 23:00

For what I understand generalization error is typically obtained from a test set, not from a training set.

In case you need to choose between several models or options, you first train your model, estimate its performance based on a second dataset (validation set), choose the best model and use it to test on a third dataset (test set) its generalization error. In case that no decisions have to be made based on the performance, two datasets (training and test sets) are enough, and generalization error comes from the test set performance as well.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.