I am unable to fully understand how this parameter works from the description in the documentation
[max_delta_step [default=0]] Maximum delta step we allow each tree’s weight estimation to be. If the value is set to 0, it means there is no constraint. If it is set to a positive value, it can help making the update step more conservative. Usually this parameter is not needed, but it might help in logistic regression when class is extremely imbalanced. Set it to value of 1-10 might help control the update
It is not clear to me what the "delta step" refers to, especially since there is already an analytical solution to the weights, and the weights are already penalized by eta
. Can someone shed some light on where this parameter fits into the algorithm, what the "delta step" refers to, and how exactly does it help in extremely imbalanced datasets?