Does it make sense to find confidence intervals for neural networks? I was trying to figure out at best the differences between confidence intervals, prediction intervals, credible intervals as tools to take into account uncertainty associated to predictions on unseen data.
I've noticed that, when we look, for example, at Linear Regression, several studies propose to compute confidence intervals to quantify the uncertainty linked to the model parameters of regression ($\beta_0, \beta_1 \, : y_i = \beta_1x_i + \beta_0$).
So for example, in this case we find an interval such that at a certain confidence, e.g. $95\%$ the parameters $\beta_0$ or $\beta_1$ will lie in that interval.
When it comes to other approaches like Neural Networks or Gradient Boosting, I have never encountered a similar approach to predict uncertainty associated to predictions but instead I've seen people running an Ensemble of models in order to come up with a distribution of the outputs and quantifying the uncertainty by taking the mean and standard deviation of outputs distribution.
So what is the difference here? This is not of course the same as considering confidence intervals, as we are now considering distributions over outputs and not over the parameters of the model.. Is this other approach related to prediction intervals or credible intervals? And why should we choose one thing instead of the other?
Many thanks for your help,
James
 A: For simple regression models, if you have the joint distribution of the parameters you get both confidence intervals and in a sort of derived fashion prediction intervals. You typically have the joint distribution for regression models, either for Bayesian models fit using MCMC samplers you have pseudo-random samples from that distribution and for maximum likelihood estimation you typically have a multivariate normal distribution approximation.
Let's start with the latter case and your example: your confidence interval for a parameter is usually something like $(\hat{\beta}_0 - 1.96 \times \text{SE}(\hat{\beta}_0),\  \hat{\beta}_0 + 1.96 \times \text{SE}(\hat{\beta}_0))$ and you have the same for $\hat{\beta}_1$. Once you want a prediction interval, you get the point prediction for a new observation with covariates $x_*$ is $\hat{\beta}_0 + \hat{\beta}_1 x_*$, but the standard error is $\sqrt{  \text{SE}(\hat{\beta}_0)^2 + x_*^2 \text{SE}(\hat{\beta}_1)^2  + 2 x_* \text{Cov}(\hat{\beta}_0, \hat{\beta}_1) + \sigma^2}$ (if we know the standard deviation $\sigma$ of the residual error term - it gets a little more complicated if we estimate that, too). So, one difference is that prediction intervals is that they take the variation in outcomes (from the residual error term) into account, too. Both are useful, because we are interested in the uncertainty of predictions, but possibly also in interpreting individual coefficients and seeing how much they might vary through sampling variation.
Similarly, for a Bayesian model you get credible intervals from the $K$ MCMC samples by considering the distribution of samples $\hat{\beta}^{(k)}_0$ for $k=1,\ldots,K$. You get a confidence interval for the linear prediction term via the distribution of $\hat{\beta}^{(k)}_0 + \hat{\beta}^{(k)}_1 x_*$ and the prediction interval via sampling for each $k$ from a $N(\hat{\beta}^{(k)}_0 + \hat{\beta}^{(k)}_1 x_*, \hat{\sigma}^{(k)})$ distribution (repeatedly or just once, as you wish). You might say that this is a lot easier and more straightforward that the frequentist case, especially taking the uncertainty around the estimated residual standard deviation is trivial.
For neural networks, gradient boosted trees etc., I don't think a CI for an individual model parameter / weight / tree split is really useful, even if you can calculate it. We just typically have a lot of trouble interpreting individual parameters, but rather tend to look at the influence of features of an input on the output. I guess you could get confidence intervals for something like SHAP values (probably just by bootstrapping), but I've indeed never seen that. What people are much more (only?) interested in is prediction intervals. Ideas for getting them include:

*

*in theory (in practice only for super simple cases), you can do the same things as above, but the complexity usually makes this challenging

*ensemble based methods (as you mentioned - one variant of that is leaving dropout on at inference time in neural networks trained with dropout)

*bootstrapping (obviously rather time consuming)

*quantile regression (e.g. your neural network has three outputs: a point prediction and, say, the 10th and 90th percentile of the distribution for points with such covariates, fit using, say, quantile regression loss of some form / pinball loss - see e.g. this discussion on a Kaggle competition)

There's probably quite a few more approaches.
