I have a dataset with mostly financial variables (120 features, 4k examples) which are mostly highly correlated and very noisy (technical indicators, for example) so I would like to select about max 20-30 for later use with model training (binary classification - increase / decrease).
I was thinking about using random forests for feature ranking. Is it a good idea to use them recursively? For example, let's say in the first round I drop the worst 20%, second too and so on until I get the desired number of features. Should I use cross-validation with RF? (It's intuitive for me not to use CV because that's pretty much what RF does already.)
Also if I go with random forests should I use them as classifiers for the binary or regressor for the actual increase / decrease to get feature importances?
By the way, the models I would like to try after feature selection are: SVM, neural nets, locally weighted regressions, and random forest. I'm mainly working in Python.