Suppose we've trained a GBDT model with 100 trees with a fairly high learning rate. Consider two cases:
We drop the first tree in the model
We drop the last tree in the model
We then compare models performance on the train set.
I wrote that in case 1 performance will drop more than in case 2 b/c a high learning rate means less corrections are made with each new tree so presumably the last tree did not contribute that many corrections to the prediction.
The answer in the book is: In the case1 performance will drop more than in the case2. In GBDT model we have sequence of trees, each improve predictions of all previous. So, if we drop first tree — sum of all the rest trees will be biased and overall performance should drop. If we drop the last tree -- sum of all previous tree won't be affected, so performance will change insignificantly (in case we have enough trees).
My question is, is my explanation wrong? Is the learning rate irrelevant in this case? Also, the question italicizes "train set," why is this distinction important?