3
$\begingroup$

Is there any theoretical justifications for Random Forests in high dimensions? I notice the work "Uniform Convergence of Random Forests via Adaptive Concentration" which shows generalization of RF under low-dimensional settings where $d\ll n$. I was wondering if RF, or more generally, any tree-type algorithms works for high dimensional data, e.g., gene data, where we have $n = 100$ data points with $d = 10000$ dimensions.

$\endgroup$
3
$\begingroup$

Thanks for the reference ! It is very new but it looks very exciting. Theory for RF has been slow to emerge. The main difficulty is how to take into account theoretically the use of the labels in the growth process of the tree. When the labels are not taken into account, consistency is somewhat similar to Stone's theorem for nearest neighbour. Which is why the work of Scornet et al who showed consistency of random forests last year was a new and important step. However, their result was not regarding a high dimensional setting. The reference you give seems the best theoretical justification there is for RF in high dimension.

In practice, RF is used in high dimensional settings. I somehow feel like 100 data points is small but you might get something out of it nonetheless. Did you try and use it on your data ? Also, you could be more explicit in what you are trying to predict in your question.

Of course, in all things biology, quality control is a major problem that can strongly impact your results. I have obtained unrealistic performances because of QC problems, so keep an eye open for this.

A very nice adaptation of RF to GWAS data has been done in the work of Botta et al. It takes into account chromosomal distance to allow for better prediction. You can find a lengthier discussion of this work on my blog. You could try and adapt this to your problem.

$\endgroup$
  • $\begingroup$ Could you be a little more explicit about how these remarks address the request for "theoretical justifications"? $\endgroup$ – whuber Dec 13 '15 at 17:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.