Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 5 down vote favorite
2

I have a multi-output regression problem with $d_x$ input features and $d_y$ outputs. The outputs have a complex, non-linear correlation structure.

I'd like to use random forests to do the regression. As far as I can tell, random forests for regression only work with a single output, so I would have to train $d_y$ random forests - one for each output. This ignores their correlations.

Is there an extension to random forests that takes output correlations into account? Maybe something like Gaussian process regression for multi-task learning.

share|improve this question
I don't know what you mean by nonlinear correlation structure. Should I assume that you mean that th response variables have a nonlinear relationship that you want to exploit in the prediction. If so that would be a multivariate regression model and in a parametric context would require explicit modeling of the nonlinear functional relationship. If there is a nonparametric analog with trees I am not aware of it. – Michael Chernick Jul 24 '12 at 4:14
1  
here's what i've been able to dig up so far: scikit-learn.org/dev/modules/tree.html#multi-output-problems – sergeyf Jul 24 '12 at 6:13
Can you tell us more about your problem? – mbq♦ Jul 24 '12 at 11:07
1  
sure. i have high dimensional "images" (actually dI/dV spectra) of peptides. the goal is to figure out the locations & classes of the amino acids that make up the peptide. my first approach was image segmentation, but CRFs and pixel-wise random forests failed. so now, instead of saying each pixel "belongs" to one & only one amino acid (not really true), i'm assigning each pixel a relative "influence" value from nearby amino acids. this results in a $d_y$ dimensional histogram for each pixel. hence, multiple output regression! – sergeyf Jul 24 '12 at 14:53
1  
It might be a belated reply: in Crimisini et al. Decision Forests: A Unified Framework for Classification, Regression, Density Estimation, Manifold Learning they use RF in a way that might suits you for organs boundaries identification. – Simone Jan 29 at 1:29

Know someone who can answer? Share a link to this question via email, Google+, Twitter, or Facebook.

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.