I apologize for this seemingly basic question, but I was just thrust into this data science role and would like to get some advice from some experts.
I have a dataset with a small number of features (only 5) but that has many, many examples (millions). There are 30 different classes represented in this dataset. I want to build a supervised classification algorithm (a random forest) using this data.
When I run a simple random forest on the data, I get about 98% accuracy, but the problem is, 8 classes are hard to distinguish (5-40% accuracy). I want to figure out a way to make these 8 classes, and well all the classes I suppose, more distinguishable so that my classifier will have an easier time classifying them.
What are some techniques to do this? I was thinking that I could perhaps warp the dataspace somehow, but from what I see, typically that idea is performed when you have tons of features. I only have a few. So, what can I do?
Ideas I have so far:
- Linear Discriminant Analysis
- Quadratic Discriminant Analysis
- Kernel PCA
- Nonlinear PCA
Is there anything else I am missing?
I tried doing PCA and ICA, but they didn't give me good results because the data are not linearly separable.
Any other ideas?
What have you seen work in practice? Which one would you recommend?
Note: I was also thinking about clustering the data. Can I use the same transform to make my clustering easier?
Again, I apologize for this basic question, but I have nobody to ask but Google. Real human advice from experience is what I am missing and looking for here.