0
$\begingroup$

I apologize for this seemingly basic question, but I was just thrust into this data science role and would like to get some advice from some experts.

I have a dataset with a small number of features (only 5) but that has many, many examples (millions). There are 30 different classes represented in this dataset. I want to build a supervised classification algorithm (a random forest) using this data.

When I run a simple random forest on the data, I get about 98% accuracy, but the problem is, 8 classes are hard to distinguish (5-40% accuracy). I want to figure out a way to make these 8 classes, and well all the classes I suppose, more distinguishable so that my classifier will have an easier time classifying them.

What are some techniques to do this? I was thinking that I could perhaps warp the dataspace somehow, but from what I see, typically that idea is performed when you have tons of features. I only have a few. So, what can I do?

Ideas I have so far:

  1. Linear Discriminant Analysis
  2. Quadratic Discriminant Analysis
  3. Kernel PCA
  4. Nonlinear PCA
  5. Autoencoder

Is there anything else I am missing?

I tried doing PCA and ICA, but they didn't give me good results because the data are not linearly separable.

Any other ideas?

What have you seen work in practice? Which one would you recommend?

Note: I was also thinking about clustering the data. Can I use the same transform to make my clustering easier?

Again, I apologize for this basic question, but I have nobody to ask but Google. Real human advice from experience is what I am missing and looking for here.

Improve this question
$\endgroup$
2
  • $\begingroup$ Did you visually explore the 8 classes? It may be that they are hardly distinguishable at all. $\endgroup$
    – ttnphns
    Apr 24, 2020 at 18:53
  • $\begingroup$ @ttnphns Yes I did. They are very non-convex and hard to distinguish from each other. That is really the point of the whole idea I think -- is there a method for me to transform this data to make these 8 classes easier to distinguish? Either by classification or clustering? $\endgroup$
    – The Dude
    Apr 24, 2020 at 18:54

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.