I noticed that I can train models in the R package gbm that have interaction depth larger than the total number of predictors used to train the model. How is it possible that I can train a model of depth 8 using only 3 predictors? Shouldn’t I be able only train it up to depth 3? What is the algorithm doing differently?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Each node in the design tree splits the predictors space to two parts. So, a new split for an already used variable sometimes can provide performance improvement.
Consider for example, a function $f(x) = \begin{cases} 0, x_1 < 2, \\ 1, x_2 < 0, x_1 \geq 2, \\ 2, x_2 \geq 0, 2 \leq x_1 \leq 3, \\ 3, x_2 \geq 0, x_1 > 3. \end{cases}$
Such function for perfect classification requires two nodes with $x_1$.
