# Uncertainty of ANN outputs as distribution parameters

It is not an uncommon practice to train neural network models via negative log likelihood $$-\mathcal{L}(x, y_{true}, \mu, \sigma)$$ to estimate both a location ($$\mu$$) and a scale ($$\sigma$$), such that $$\mu, \sigma = f(x)$$, where $$f$$ is the optimised neural network and $$x$$ some data.

During training, the $$\mu$$ and $$\sigma$$ network outputs parameterise a distribution $$p$$, the negative log likelihood ($$\mathcal{L}$$) of which is used as a training objective. The intuition being that the neural network is trained w.r.t how well the predicted distribution explains the training sample, and thus the neural network learns to predict the distribution of the output quantity corresponding to the data point $$x$$.

However, I fail to understand how this yields valid uncertainty estimates ($$\sigma$$) at inference time for the following reasons:

1. As the $$\mu$$ and $$\sigma$$ are themselves point estimates, we do not know their associated uncertainty.
2. If we assume that during training, the per data point estimated $$\sigma$$'s yield calibrated, heteroscedastic residuals, it is not guaranteed that this will hold for out of sample data (due to covariate shift), assuming temperature scaling and suchlike have not been performed.
3. If the data are not from the distribution family of $$p$$ (and so $$\mathcal{L}$$), the predicted uncertainty $$\sigma$$ could be further miscalibrated.

So, for a neural network that predicts $$\mu$$ and $$\sigma$$, under which conditions can one consider the scale $$\sigma$$ to be a valid estimate of uncertainty when taking the location $$\mu$$ to be the predicted quantity?