Reference request: consistency of gradient boosting methods

Question

I'm looking for work done on showing the consistency of gradient boosting methods such as gradient boosted decision trees. Friedman's original work only introduces the algorithm, but does not provide any analysis of the consistency properties.

By "consistency" I mean in the sense that following the gradient boosting algorithm minimizes the expected loss as the number of samples goes to infinity (https://en.wikipedia.org/wiki/Consistency_(statistics)).

I'm also interested in convergence rate bounds under different assumptions on the distribution.

My expectation is that the results from empirical risk minimization and analyses of gradient descent apply directly to gradient boosting, but I'm wondering if it's more subtle than that. Are there sources discussing this in more detail?

Answer 1

[1] Shows consistency of the gradient boosting algorithm and gives some background on related work.

[2] Extends gradient boosting to the case of nesterov momentum and gives bounds on the optimization rate (how fast does the excess empirical risk go to zero).

References

[1] Biau, G., Cadre, B. (2017). Optimization by gradient boosting arXiv https://arxiv.org/abs/1707.05023

[2] Lu, H., Karimireddy, S., Ponomareva, N., Mirrokni, V. (2019). Accelerating Gradient Boosting Machine arXiv https://arxiv.org/abs/1903.08708