I'm trying to get a handle on what the current state of things is when it comes to quantifying uncertainty in nonparametric regression models. It seems like the options are
- Use a Bayesian model and hope the model fits well enough to be able to trust the posterior predictive distributions it comes up with
- Use a non-Bayesian model for which there are known uncertainty quantification techniques, e.g. random forest + jackknife
- Use a generally applicable trick that can be applied to any model, e.g. bootstrap confidence intervals, conformal prediction
- Try to model the residuals of your main model so that you can work out something like the "expected error" of the main model on a new datum
I have a few questions:
- Have I overlooked any potential approaches in the above list?
- Are there other good, flexible regression methods that people have figured out uncertainty quantification for? Random forests are the only such method I've come across.
- Are people using bootstrap confidence intervals/conformal prediction on large datasets in production? I haven't managed to find much on this.
I realise that this is a broad question but hope that's ok. Thanks!