Return to Answer

XGBoost uses a diagonal approximation to the Hessian. A diagonal $n\times n$ matrix has at most $n$ nonzero elements. The diagonal approximation scales nicely, because it only grows linearly in $n$, as opposed to the dense Hessian which grows quadratically.

The diagonal approximation is the best when the off-diagonal elements are close to zero. However, even when the off-diagonal elements are far from zero, the reasoning is that this is tolerable because the purpose of subsequent boosting rounds is to reduce model misfit. (For an alternative perspective, consider that another group of researchers proposes an improved diagonal approximation and found that this yields better results. "A Fast Sampling Gradient Tree Boosting Framework" by Daniel Chao Zhou, Zhongming Jin, Tong Zhang.)

XGBoost uses a diagonal approximation to the Hessian. A diagonal $n\times n$ matrix has at most $n$ nonzero elements. The diagonal approximation scales nicely, because it only grows linearly in $n$, as opposed to the dense Hessian which grows quadratically.

The diagonal approximation is the best when the off-diagonal elements are close to zero. However, even when the off-diagonal elements are far from zero, the reasoning is that this is tolerable because the purpose of subsequent boosting rounds is to reduce model misfit. (For an alternative perspective, consider that another group of researchers proposes a modified diagonal approximation and found that this yields better results. "A Fast Sampling Gradient Tree Boosting Framework" by Daniel Chao Zhou, Zhongming Jin, Tong Zhang.)

XGBoost uses a diagonal approximation to the Hessian. A diagonal $n\times n$ matrix has at most $n$ nonzero elements. The diagonal approximation scales nicely, because it only grows linearly in $n$, as opposed to the dense Hessian which grows quadratically.

The diagonal approximation is the best when the off-diagonal elements are close to zero. However, even when the off-diagonal elements are far from zero, the loss of precision isn't an enormous concern because the purpose of subsequent boosting rounds is to reduce model misfit.

XGBoost uses a diagonal approximation to the Hessian. A diagonal $n\times n$ matrix has at most $n$ nonzero elements. The diagonal approximation scales nicely, because it only grows linearly in $n$, as opposed to the dense Hessian which grows quadratically.

The diagonal approximation is the best when the off-diagonal elements are close to zero. However, even when the off-diagonal elements are far from zero, the reasoning is that this is tolerable because the purpose of subsequent boosting rounds is to reduce model misfit. (For an alternative perspective, consider that another group of researchers proposes an improved diagonal approximation and found that this yields better results. "A Fast Sampling Gradient Tree Boosting Framework" by Daniel Chao Zhou, Zhongming Jin, Tong Zhang.)

XGBoost uses a diagonal approximation to the Hessian. A diagonal $n\times n$ matrix has at most $n$ nonzero elements. The diagonal approximation scales nicely, because it only grows linearly in $n$, as opposed to the dense Hessian which grows quadratically.

The diagonal approximation is the best when the off-diagonal elements are close to zero. However, even when the off-diagonal elements are far from zero, the loss of precision isn't an enormous concern because the purpose of subsequent boosting rounds is to reduce model misfit.

XGBoost uses a diagonal approximation to the Hessian. A diagonal $n\times n$ matrix has at most $n$ nonzero elements. The diagonal approximation scales nicely, because it only grows linearly in $n$, as opposed to the dense Hessian which grows quadratically.

The diagonal approximation is the best when the off-diagonal elements are close to zero. However, even when the off-diagonal elements are far from zero, the loss of precision isn't an enormous concern because the purpose of subsequent boosting rounds is to reduce model misfit.