1
$\begingroup$

I was checking out how to create prediction intervals with Gradient boosted regression trees using Scikit-learn. If you set the alpha at .95 or .05, you can get the 95% prediction interval around the prediction.

When I tried running it on my own data, I realized that sometimes, or often, the predicted mean was ABOVE the upper 95% prediction interval. How is this possible? It happens whether I change the loss function to 'ls' or keep it at 'quantile' but change the alpha to .5.

For an example, look at sklearn's documentation on it. You will notice that one of their predictions sits outside of the prediction intervals.

Note - it is not the observed value that goes beyond the prediction interval (which would be sensible ~5% of the time) but the prediction for that value of x. Said another way, if the quantile is set at .5 the predicted value, the prediction is sometimes higher than when the quantile is set at .95. That doesn't make sense to me.

http://scikit-learn.org/stable/auto_examples/ensemble/plot_gradient_boosting_quantile.html

Can anyone explain this to me?

Also, should a loss function of 'ls' == a loss function of 'quantile' with an alpha =.5?

$\endgroup$
2
$\begingroup$

I can answer the last part of your question: The 'ls' loss is different to the 'quantile' loss with an alpha of 0.5. While the former yields an estimate for the conditional mean of your response variable (through minimizing the sum of the residual least squares), the latter yields an estimate for the 50% quantile, i.e. the conditional median.

Regarding the first part of your question, maybe this paper helps (they propose a modified version of quantile regression which avoids the crossing quantiles you mentioned): http://www4.stat.ncsu.edu/~hdbondel/noCross.pdf

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.