0
$\begingroup$

My data set is divided into 80:20 train and test...i have performed 10 fold cross validation on the train data set and tested the 20 % dataset on each iteration ( so that test set is not touched while training). Finally i get the scores by averaging the scores in each iterations.
I have been trying classification on a 7 class problem. I have 8 sensors generated data .(8features). Every time the classifier misclassifies the last class. I tried decreasing the number of classes, still the last class got miss classified.
Finally i started decreasing the test set to increase training. I got good results (90%accuracy) when test data is only 8%.

Is any other way around or any scope of increasing scores without further decreasing the size of the test set? following are the snipets of the two casesenter image description here

$\endgroup$
  • $\begingroup$ How does the distribution of your classes look? Have you examined the class distributions in the entire sample and then in such smaller subsamples? $\endgroup$ – deemel Oct 27 '19 at 11:26
  • $\begingroup$ The classes are distributed almost equally, plus-minus one or two raws between the subsamples $\endgroup$ – Sudipt Oct 27 '19 at 12:38
  • $\begingroup$ How is the accuracy in both cases per class? It could of course be that due to a smaller training partition resulting from a bigger test partition, the fitted model generalizes worse than with a bigger training partition, but this depends strongly on the problem and the sample size we're talking about. On the other hand, it might as well be that your smaller test sets merely present a more 'accomodating' sample for your model. This is something we cannot really answer without speculation. You could investigate both angles and report what you find, maybe then the picture gets clearer. $\endgroup$ – deemel Oct 27 '19 at 13:40
  • $\begingroup$ i have added a confusion matrix and some scores as an image. Regarding your view on training and test partitions, i will check and surely report. thanks $\endgroup$ – Sudipt Oct 27 '19 at 13:58
0
$\begingroup$

Generally speaking, with more training data, the model will learn the underlying distribution of the real data better. Since a larger training set in your case improves the performance of on training and test set, you should get more data if you can. The performance on the test set might not be that reliable if your test set is small. In other words, your performance might be different if you change to another test set. That is one of the reasons that you perform cross validation. I suggest you also take a look at the CV accuracy.

You should also take a look at the class distribution in your training and test set. If there are only a few data points for the last class, the classifier will not able to learn well. You can upsample the minority class to make the classifier work better.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.