Machine learning for multi-level response

Question

I have a dataset with ~90000 observations and less than 10 features (all continuous). The problem is that the response variable has ~300 categories. Currently I would try to fit a multinomial linear model or a random forest. I have to predict probabilities for the 300 categories. I need some inspiration and would like to know which model others would use?

Answer 1

Since you say you are interested in probabilities (and not just prediction) this will impact your choice of algorithm. You really have to choices:

1. Use a classifier that predicts well-calibrated probabilities.
2. Use any multiclass classifier and attempt to calibrate the probabilities using something like Platt scaling or isotonic regression, see section 2.2 of the paper Calibrating Random Forest for a brief description of the techniques and related references.

I would probably start with the first option using Multinomial Logistic Regression and then maybe a Neural Network if I felt I was underfitting. As someone said in a comment, random forests seem like a good option, but you'll likely need to calibrate them somehow (using the technique in the first paper I linked?). Another good reference for getting probabilities out of supervised classifiers is the aptly titled Predicting Good Probabilities With Supervised Learning.

Answer 2

When you are dealing with many class problems like this, some of the classes will likely be not well represented. If you are really interested in getting these "not-well-represented" classes right, aka increasing the number of True Positives but generating a lot of False Positives, you may need to tune your model. In Random Forest you will need to play with class weights or sampling strategies that up sample the not well represented classes.

K-Nearest-Neighbors is another potential classifier.