I am a little confused about this situation: I have a dataset for classification problem. It is divided into a training and a testing set. And I've used the training set to generate an artificial dataset from it (using an oversampling technique) to get a more balanced dataset (because the ratio between the two classes was very y unbalanced). I want to test my artificial dataset. So I want to use some classifiers such as Decision Tree, to compare with the orginal datasets.

The thing is that I am training the classifier with the artificial data, and then using that model I provide the testing dataset to get the prediction (as a final step: I get a confusion matrix); and also using the training dataset to predict and get another confusion matrix but in this case, as validation values. Is this approach OK? I am doing it well? should I training the model using the training dataset or the artificial?

Sorry if this is a little dum question, but I am new in this, and also I've seen only training and testing steps, and not the validation of models...

Thank you!!


closed as unclear what you're asking by Michael Chernick, kjetil b halvorsen, Peter Flom May 7 '18 at 12:23

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • $\begingroup$ I don't think you should be creating an artificial data set here. $\endgroup$ – Peter Flom May 7 '18 at 12:23

You have started out on the wrong foot IMHO. Split sample validation requires huge sample sizes and so do single tree methods. You may have miscast the entire problem as a classification problem instead of a prediction problem. Use of oversampling means that you don't understand the statistical methods behind the method, and that you think that it is legal to make up data. Any method that requires over/undersampling is defective. Details are here and here.


Not the answer you're looking for? Browse other questions tagged or ask your own question.