2
$\begingroup$

I am using J48 decision tree classifier in weka. In the testing option I am using percentage split as my preferred method. The split use is 70% train and 30% test. My understanding is that when I use J48 decision tree, it will use 70 percent of my set to train the model and 30% to test it. Although it gives me the classification accuracy on my 30% test set, I am confused as to why the classifier model is built using all of my data set i.e 100 percent. Shouldn't it build the classifier model only on 70 percent data set?

It mentions in the classification window that === Classifier model (full training set) === However, when I check the decision tree , it uses all 100 percent data instead of 70? Is there a particular reason why Weka does this?

$\endgroup$
  • $\begingroup$ Normally the trees are fit on the training data only. I see why you might be puzzled. I am not familiar with Weka and J48. $\endgroup$ – Michael R. Chernick Apr 14 '17 at 21:09
1
$\begingroup$

WEKA builds more than one classifier. It displays the one built on all of the data but uses the 70/30 split to predict the accuracy. For this reason, in most cases, the accuracy of the tree displayed does not agree with the reported accuracy figure. The reported accuracy (based on the split) is a better predictor of accuracy on unseen data.

$\endgroup$
  • $\begingroup$ I still don't understand as to why display a classifier model using " all data set" then. Is it a standard practice in machine learning to report model based on all data? $\endgroup$ – Shalin Parikh Apr 14 '17 at 22:17
  • 1
    $\begingroup$ Yes, the model based on all data uses all of the information and so probably gives the best predictions. $\endgroup$ – G5W Apr 14 '17 at 23:20
  • 1
    $\begingroup$ (+1) The idea is that fitting the model to 70% of the data is similar enough to fitting it to all the data for the performance of the former procedure in predicting for the remaining 30% to be a decent estimate of the performance of the latter in predicting for unseen data. A still better estimate would be got by repeating the whole process for different 30%s & taking the average performance - leading to the technique of cross validation (q.v.). $\endgroup$ – Scortchi - Reinstate Monica Aug 10 '18 at 8:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.