The goal is to train some classifiers to achieve decent classification accuracy, in order to find out what are the features that are most important.
The random forest classifier is being used to train on a dataset that is split to train and test sets randomly. Due to the relatively small dataset size compared to number of features, there is large variation in the set of important features between different random training & test set. E.g. train-test split 1 will give very different feature importance ranking compared to train-test split 2.
My questions would be:
- Is doing multiple runs of the same dataset but split differently by random, an acceptable practice?
- If so, what's the good or common practice for combining the results of different runs?