So basically I have a machine learning model where I want to have a prediction interval, the model is XGBoost so it is tricky to do Quantile Regression and I was looking for an alternative method to achieve this goal.
I have came across this article that suggests to estimate the standard deviation for the prediction by creating another machine learning model to predict the error of the original machine learning model. So the process would be:
- Fit machine learning model to training data.
- Calculate the error for each datapoint.
- Fit a second machine learning model to predict the squared error of the datapoints.
- When making a prediction use the initial prediction as mean and the square root of the prediction of the squared error as an estimate of the standard deviation.
- With the mean and std deviation build a confidence interval for the prediction.
The assumptions here is that the distribution of the predictions is Normal and that we can estimate the mean and standard deviation in the way mentioned.
My question is, does this method make sense? And is there a better way to do this?