Likelihood that a prediction falls above (below) 110% (90%) of the prediction

Question

For my client I have to predict some products' prices with gbm (scikit). So in the production, I am to give prediction intervals. That is, I need to provide how likely a real price falls above 110% or falls below 90% of the corresponding prediction (i.e. if prediction is 100$ then what is the probability that the real price falls >= 110 or <= 90 or within 90-110). And the decision will be to trust the model if the likelihood of being in the 90%-110% interval is >=90%.

I am bit confused from what I read and would like to learn some important points.

• Would I be able to get what I want with https://scikit-learn.org/stable/auto_examples/ensemble/plot_gradient_boosting_quantile.html? i.e. with setting alpha to 0.90 for example, I score a new point with upper & lower fit and if my prediction * 110% (and 90%) cover these 2 thresholds, then I would interpret that there is at least 90% likelihood that real value will be between pred90% - pred110? Could I get separate probabilities i.e. prob of below pred90% and prob of pred110%?

• How can I do this with bootstrap (again for only gbm model)? I see that I can get prediction intervals as shown here https://saattrupdan.github.io/2020-03-01-bootstrap-prediction/. However, I am confused with how I do what I want with a totally new observation in production.

• what are the other ways to achieve this (get all 3 probabilities)?

Any guidance would be appreciated so much!

Answer 1

I don't think that link from scikit learn gives you want you want. That method only produces the quantiles, not any probabilities you see prices that extreme. If I've understood correctly, you want to know the probability that you see price that is over \$55 and under \$45 for something that is priced \$50. The method you linked would give you prices for which there is a $\alpha\%$ chance of seeing a price larger/smaller.

In order to get probabilities of the type you mention, I think you need to assume a likelihood for the data (that is, make an assumption on the distribution of the data, conditional on all the things you predict). From there, you can simulate some fake data and compute tail probabilities that new data are observed beyond those limits. Here is an example I did with Gamma Regression.

Alternatively, you could do some bisection method whereby you find a quantile which happens to be \$55 and \$45 for something priced \$50. Once you find the quantile, the probability you want would be straightforward.