What model to use: Feature importance at each rank

My dataset contain a set of ordinal features and a rank from 1 to 10 which I'm trying to predict.

I'm looking for a way to understand which feature is most important at each rank.

Which model is best to use to get this information?

Would learning-to-rank models (RankSVM ...) work in this case? I'm not dealing with queries in this case.

1 Answer

I don't know learning to rank models that well so I will not touch on those, however you could also treat this as a multi-class problem.

If you are using sklearn then you can use a one-vs-rest strategy where you train one classifier for each "label" which in this case would be 10 classifiers, one for each rank. Every sample is then evaluated against each classifier and is assigned the label that gets the highest probability.

The big advantage for you is that since there is one and only one classifier per label you will be able to inspect coefficients or feature importances to figure out which features are most important at each rank.

While a learning to rank approach might give you better accuracy this will at least give you a starting point which is quick to set up and which results in easily interpretable models.