3
$\begingroup$

I noticed that I can train models in the R package gbm that have interaction depth larger than the total number of predictors used to train the model. How is it possible that I can train a model of depth 8 using only 3 predictors? Shouldn’t I be able only train it up to depth 3? What is the algorithm doing differently?

$\endgroup$
5
$\begingroup$

Each node in the design tree splits the predictors space to two parts. So, a new split for an already used variable sometimes can provide performance improvement.

Consider for example, a function $f(x) = \begin{cases} 0, x_1 < 2, \\ 1, x_2 < 0, x_1 \geq 2, \\ 2, x_2 \geq 0, 2 \leq x_1 \leq 3, \\ 3, x_2 \geq 0, x_1 > 3. \end{cases}$

Such function for perfect classification requires two nodes with $x_1$. enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.