# Algorithm for finding all local maxima of a boosted regression tree?

Given a weighted sum of regression trees, is there an efficient algorithm to find all local maxima ?

(I would tend to think that a gradient based method will find some maxima but it is not entirely clear to me how to ensure that one will find all of them)

-
Two things: 1) A sum of regression trees is piecewise constant, so I don't think it has a gradient (it's flat everywhere that it's continuous, so I don't think gradient methods are an option). 2) Maybe if you clarified why you want the maxima we'd be able to help you better? –  David J. Harris Apr 30 '12 at 21:04
1) yes, I'm aware that this is actually a piecewise constant. So I was thinking of the brute force approach of investigating all $O(N^d)$ boxes where $N$ is a typical number of regions per variable (taking into account all trees) and $d$ is the number of variables. With $N=100$ and $d=10$ this is intractable however. 2) I want to get a better understanding of what the overall function is doing by looking at regions of local maxima (or more generally extrema) and examine by hand whether these maxima make sense or not. –  Andre Holzner May 1 '12 at 15:56
This doesn't answer your question directly, but there are alternatives to boosted trees called "bump-hunting" approaches (example), which automatically identify maxima as part of the model (see especially sections 5 and 15 of the linked paper). There are even boosted bump-hunting methods, which may get better accuracy. –  David J. Harris May 1 '12 at 19:37