Post hoc selection of important features in random forest? I want to guarantee a parsimonious random forest (few features used). What are methods to do this? It was suggested to me to get the feature importance after the model was created, and then create a new model using only the important features. Is this a good idea/what other options are there?
 A: Filtering out low importance features is a common idea but you need to do it iteratively and only remove a (tuned) percentage features at each iteration. This is because importance in rf can be spread over multiple features with redundant information.
Ie if you have both "predictor_a" and "transformed_predictor_a" in your data set it will tend to use them interchangeably thus reducing the importance of both by half. Iteratively removing features will (hopefully) only remove the worse of the two features and the other will get an importance boost preventing it from being removed in future iterations.
There are also methods that attempt to deal with this importance spreading analytically by looking at "masking scores" to determine information redundancy or by doing other additional analysis.
The other part of the method described in the linked paper above can also be quite effective. It computes multiple forests each shuffled copies of all of the data set and does a statistical test to determine which of the features perform significantly better then their contrasts.
In my experience this method (ace feature selection) can work extremely well in very high dimensional data sets with lots of irrelevant noisy features, including especially genetic data. I have an implementation and a plot with some results here. 
A: I'm working (on my own) with a cousin of de-boosted trees.  
Boosting increases sensitivity and characterization, but I am wanting a robust fit.  The approach that I currently have has results that appear similar to ridge regression, but in test sets I can get good results.  Another reason that it isn't a proper GBT is that I replace the tree by making a weighted mix of the errors from the new "tree" and the "old".  The weight works like a learning parameter and should be small.  The result is a robustly fit Classification and Regression Tree (CART) model.  
In some senses it qualifies as optimally parsimonious.  In the sense of a random forest it is optimally parsimonious because it is a forest comprised of the least nontrivial count of trees - one.
There is an interesting connection between parsimony and robustness when it comes to CART models.
