What does interaction depth mean in GBM? I had a question on the interaction depth parameter in gbm in R. This may be a noob question, for which I apologize, but how does the parameter, which I believe denotes the number of terminal nodes in a tree, basically indicate X-way interaction among the predictors? Just trying to understand how that works. Additionally, I get pretty different models if I have a dataset with say two different factor variables versus the same dataset except those two factor variables are combined into a single factor (e.g. X levels in factor 1, Y levels in factor 2, combined variable has X * Y factors). The latter is significantly more predictive than the former. I had thought increasing interaction depth would pick this relationship up.
 A: Both of the previous answers are wrong. Package GBM uses interaction.depth parameter as a number of splits it has to perform on a tree (starting from a single node). As each split increases the total number of nodes by 3 and number of terminal nodes by 2 (node $\to$ {left node, right node, NA node}) the total number of nodes in the tree will be $3*N+1$ and the number of terminal nodes $2*N+1$. This can be verified by having a look at the output of pretty.gbm.tree function.
The behaviour is rather misleading, as the user indeed expects the depth to be the depth of the resulting tree. It is not.
A: 
I had a question on the interaction depth parameter in gbm in R. This may be a noob question, for which I apologize, but how does the parameter, which I believe denotes the number of terminal nodes in a tree, basically indicate X-way interaction among the predictors?

Link between interaction.depth and the number of terminal nodes
One as to see interaction.depth as the number of split nodes. An interaction.depth fixed at k will result in nodes with k+1  terminal nodes (omitting the NA nodes), so we have : 
$$interaction.depth=\#\{Terminal Nodes\}+1 $$
Link between interaction.depth and the interaction order
The link between interaction.depth and interaction order is more tedious.
Instead of reasoning with the interaction.depth, let's reason with the number of terminal nodes, which we will called J. 
Example:
Let's say you have J=4 terminal nodes (interaction.depth=3) you can either :


*

*do the first split on the root, then the second split on the left node of the root and the third split on the right node of the root. The interaction order for this tree will be 2.

*do the first split on the root, then the second split on the left (respectively right) node of the root, and a third split on this very left (respectively right) node. The interaction order for this tree will be 3.


So you cannot know in advance what will be the interaction order between your features in a given tree. However it is possible to upper bound this value. Let P be the interaction order of the features in a given tree. We have :  $$P\leq min(J-1,n)$$ with n being the number of observations. For more details see the section 7 of the original article of Friedman. 
A: Previous answer is not correct. 
Stumps will have an interaction.depth of 1 (and have two leaves). But interaction.depth=2 gives three leaves.
So:
NumberOfLeaves = interaction.depth + 1
A: Actually, the previous answers are incorrect.
Let K be the interaction.depth, then the number of nodes N and leaves L (i.e terminal nodes) are respectively given by the following: 
$$\begin{align*}
N &= 2^{(K+1)} - 1\\
L &= 2^K 
\end{align*}
$$
The previous 2 formulas can easily be demonstrated: a tree of depth K can be seen as having K+1 levels k ranging from 0 (root level) to K (leaf level). 
Each of these levels has $2^k$ nodes. And the tree's total number of nodes is the sum of the number of nodes at each level. 
In mathematical terms: 
$$
N = \sum_{k=0}^K 2^k)
$$
which is equivalent to:
$$N = 2^{(K+1)} - 1 $$(as per the formula of the sum of the terms of a geometrical progression).
A: You can try 
table(predict(gbm( y ~.,data=TrainingData, distribution="gaussian", verbose =FALSE, n.trees =1 , shrinkage =0.01, bag.fraction =1 ,
interaction.depth = 1 ),n.trees=1))
and see that there are only 2 unique predicted values.  interaction.depth = 2 will get you 3 distinct predicted values.  And convince yourself. 
