Combining prediction intervals

Question

I'm using ML regressors (neural networks and random forests) to predict some numbers. I can put in my inputs and get out a value and its prediction interval. The inputs to my regressors are noisy, however, so I vary the inputs within their uncertainties in a monte carlo fashion and obtain a sample of prediction intervals. How can I combine them to get one big prediction interval?

Answer 1

If your "raw" prediction intervals are simply quantiles of Monte Carlo outcomes of the separate methods, the simplest way would be to just throw all the MC outcomes from all methods together, then take quantiles of this "total" set of outcomes.

You could even weight the outcomes from separate methods, if you happen to trust one method more than another, then take quantiles from weighted data.

Apart from that, there is quite some literature on "combining interval forecasts", e.g. this. Alternatively, you could look at combining density forecasts (which is really what I am proposing above, and what you seem to be doing), which is somewhat different. Here is a relevant paper by Hall & Mitchell (2007) in the International Journal of Forecasting.