Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 0 down vote favorite

Given a weighted sum of regression trees, is there an efficient algorithm to find all local maxima ?

(I would tend to think that a gradient based method will find some maxima but it is not entirely clear to me how to ensure that one will find all of them)

share|improve this question
Two things: 1) A sum of regression trees is piecewise constant, so I don't think it has a gradient (it's flat everywhere that it's continuous, so I don't think gradient methods are an option). 2) Maybe if you clarified why you want the maxima we'd be able to help you better? – David J. Harris Apr 30 '12 at 21:04
1) yes, I'm aware that this is actually a piecewise constant. So I was thinking of the brute force approach of investigating all $O(N^d)$ boxes where $N$ is a typical number of regions per variable (taking into account all trees) and $d$ is the number of variables. With $N=100$ and $d=10$ this is intractable however. 2) I want to get a better understanding of what the overall function is doing by looking at regions of local maxima (or more generally extrema) and examine by hand whether these maxima make sense or not. – Andre Holzner May 1 '12 at 15:56
1  
This doesn't answer your question directly, but there are alternatives to boosted trees called "bump-hunting" approaches (example), which automatically identify maxima as part of the model (see especially sections 5 and 15 of the linked paper). There are even boosted bump-hunting methods, which may get better accuracy. – David J. Harris May 1 '12 at 19:37

Know someone who can answer? Share a link to this question via email, Google+, Twitter, or Facebook.

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.