# How to determine the confidence of a neural network prediction?

To illustrate my question, suppose that I have a training set where the input has a degree of noise but the output does not, for example;

# Training data
[1.02, 1.95, 2.01, 3.06] : [1.0]
[2.03, 4.11, 5.92, 8.00] : [2.0]
[10.01, 11.02, 11.96, 12.04] : [1.0]
[2.99, 6.06, 9.01, 12.10] : [3.0]


here the output is the gradient of the input array if it were noiseless (not the actual gradient).

After training the network, the output should look something like this for a given input.

# Expected Output
[1.01, 1.96, 2.00, 3.06] : 95% confidence interval of [0.97, 1.03]
[2.03, 4.11, 3.89, 3.51] : 95% confidence interval of [2.30, 4.12]


My question is how can a neural network be created such that it will return a predicted value and a measure of confidence, such as a variance or confidence interval?

• If you are looking for an interval that will contain a future realization, then you are looking for a prediction-interval, not a confidence-interval, which pertains to unobservable parameters. This is often confused. Nov 24, 2016 at 8:47

It sounds like you are looking for a , i.e., an interval that contains a prespecified percentage of future realizations. (Look at the tag wikis for and for the difference.)

Your best bet is likely to work directly with NN architectures that do not output single point predictions, but entire predictive distributions. You can then directly extract desired prediction intervals (or mean, or median point predictions) from these distributions. I and others have been arguing that predictive distributions are much more useful than point predictions, but to be honest, I have not yet seen a lot of work on predictive distributions with neural nets, although I have been keeping my eyes open. This paper sounds like it might be useful. You might want to search a bit, perhaps also using other keywords like "forecast distributions" or "predictive densities" and such.

That said, you might want to look into Michael Feindt's NeuroBayes algorithm, which uses a Bayesian approach to forecast predictive densities.

• This may be another useful paper - a neural net that learns distributions: google.com/… Apr 12, 2017 at 2:54
• @Stephan: The link has passed away : ( Apr 27, 2018 at 21:28
• @MatthewDrury: which link do you mean? All three are working fine for me. Apr 29, 2018 at 6:11
• Can you lead us to a simple Ternsorflow demo/example with NN Predictive Distributions? Jun 25, 2019 at 10:43
• @MartinThøgersen: sorry, no, I don't use Tensorflow... Jun 25, 2019 at 10:46

I'm not sure you can compute a confidence interval for a single prediction, but you can indeed compute a confidence interval for error rate of the whole dataset (you can generalize for accuracy and whatever other measure you are assessing).

If $e$ is your error rate while classifying some data $S$ of size $n$, a 95% confidence interval for your error rate is given by: $$e \pm 1.96\sqrt{\frac{e\,(1-e)}{n}}$$.

(see "Machine Learning" book from Tom Mitchell, chapter 5.)

EDIT

Guess I should state a more general case, which is: $$e \pm z_N\sqrt{\frac{e\,(1-e)}{n}},$$ where common choices for $z_N$ are listed in the following table:

confidence level    80%    90%    95%    98%    99%
values of zN       1.28   1.64   1.96   2.33   2.58

• This would require that the asymptotic distribution is normal May 1, 2017 at 16:45
• For large samples sizes (which is quite common in ML) it is generally safe ti assume that. There was no need ti downvote, just ask for clarification, but oh well.
– mp85
Jun 1, 2017 at 10:02

In terms of directly outputting prediction intervals, there's a 2011 paper 'Comprehensive Review of Neural Network-Based Prediction Intervals'

They compare four approaches:

1: Delta method 2: Bayesian method 3: Mean variance estimation 4: Bootstrap

The same authors went on to develop Lower Upper Bound Estimation Method for Construction of Neural Network-Based Prediction Intervals which directly outputs a lower and upper bound from the NN. Unfortunately it does not work with backprop, but recent work made this possible, High-Quality Prediction Intervals for Deep Learning.

Alternative to directly outputting prediction intervals, Bayesian neural networks (BNNs) model uncertainty in a NN's parameters, and hence capture uncertainty at the output. This is hard to do, but popular methods include running MC dropout at prediction time, or ensembling.

I don't know of any method to do that in an exact way.

A work-around could be to assume that you have gaussian noise and make the Neural Network predict a mean $\mu$ and variance $\sigma$. For the cost function you can use the NLPD (negative log probability density). For datapoint $(x_i,y_i)$ that will be $-\log N(y_i-\mu(x_i),\sigma(x_i))$. This will make your $\mu(x_i)$ try to predict your $y_i$ and your $\sigma(x_i)$ be smaller when you have more confidence and bigger when you have less.

To check how good are your assumptions for the validation data you may want to look at $\frac{y_i-\mu(x_i)}{\sigma(x_i)}$ to see if they roughly follow a $N(0,1)$. On test data you again want to maximize the probability of your test data so you can use NLPD metric again.

• @D.W. no because as $\sigma \rightarrow +\infty$ the distribution starts to ressemble a uniform with 0 density at all points. Then, the probability density of your datapoints goes to 0 and thus its log goes to infinity, which makes the loss go to infinity.
– etal
Aug 17, 2017 at 22:25
• Are there any concrete examples anyone's seen of using a NN to output parameters of a distribution, trained over the log likelihood? Oct 20, 2017 at 14:15

Prediction intervals (PI) in non parametric regression & classification problems, such as neural nets, SVMs, random forests, etc. are difficult to construct. I'd love to hear other opinions on this.

However, as far as I know, Conformal Prediction (CP) is the only principled method for building calibrated PI for prediction in nonparametric regression and classification problems. For a tutorial on CP, see Shfer & Vovk (2008), J. Machine Learning Research 9, 371-421 [pdf]

I have not heard of any method that gives a confidence interval for a neural network prediction. Despite a lack of formal methodology, it seems like it might be feasible to construct one. I have never attempted this due to the compute power that would be needed and I make no claims on this working for certain, but one method that might work for a tiny neural net (or with blazing fast GPU power it could work for moderate sized nets) would be to resample the training set and build many similar networks (say 10,000 times) with the same parameters and initial settings, and build confidence intervals based on the predictions for each of your bootstrapped net.

For example, in the 10,000 networks trained as discussed above, one might get 2.0 (after rounding the neural net regression predictions) 9,000 of those times, so you would predict 2.0 with a 90% CI. You could then build an array of CIs for each prediction made and choose the mode to report as the primary CI.

• I'd be curious why this suggestion was down voted as it is essentially bootstrapping in a slightly unconventional way (the rounding component of the problem makes it easy to check how confident the neural net is about the prediction). I don't actually mind the down vote if whoever down voted this could explain why this is not a valid solution to the question proposed. I'm learning myself and would appreciate feedback! Nov 25, 2016 at 3:15
• I didn't vote down, but from what I understand the proposed method would output intervals that capture the model's predicted values, this is not the same as intervals that capture the true values. Oct 20, 2017 at 14:10

There are actually ways of doing this using dropout. Run the evaluation with dropout enabled (it's usually disabled for evaluation but turned on when training), and run the evaluation several times.

The result distribution from multiple different runs can be used as confidence intervals.

See the paper "Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning" Watch the youtube presentation Andrew Rowan - Bayesian Deep Learning with Edward (and a trick using Dropout)

http://mlg.eng.cam.ac.uk/yarin/blog_3d801aa532c1ce.html

Work in Progress...

There has been a lot of work on predictive intervals for neural nets going back over the years:

The simplest approach (Nix and Weigend, 1994) is to train a second neural network to predict the mean-squared error of the first. Regression networks trained to minimise the mean-squared error learn the conditional mean of the target distribution, so the output of the first network is an estimate of the conditional mean of the targets and the second learns the conditional mean of the squared distance of the targets from the mean, i.e. the conditional variance. In practice, they don't have to be separate networks, you can have one network with two outputs, one for the conditional mean and one for the conditional variance.

References

D. A. Nix and A. S. Weigend, "Estimating the mean and variance of the target probability distribution," Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94), 1994, pp. 55-60 vol.1, doi: 10.1109/ICNN.1994.374138.

David A. Nix, Andreas S. Weigend, Learning Local Error Bars for Nonlinear Regression, NIPS 1994 (pdf)

CM Bishop, CS Qazaz, Regression with input-dependent noise: A Bayesian treatment, Advances in neural information processing systems, 347-353, 1997 (pdf)

There is no way, all ML models is not about phenomen understanding, it's interpolation methods with hope "that it works". Start with such questions confidence, robustnes to noise there is no answers.

So to derive something please use various applied and fundamental science:

• Use control (and make assumption about dynamics)

• Use convex optimization (with some extra condition on function)

• Use math statistics (with preliminary assumptions on distributions)

• Use signal processing (with some assumptions that signal is band limited)

Scientist use some prelimiary assumptions (called axioms) to derive something.

There is no way to give any confidence without some preliminary assumption, so problem in not in DL mehtod, but it's problem in any method which try to interpolate without ANY preliminary assumption-there is no way to derive via algebra something intellegently without an assumption.

NN and various ML methods are for fast prototyping to create "something" which seems works "someway" checked with cross-validation.

Even more deeper the regression fitting E[Y|X] or it's estimate can be absolutely incorrect problem to solve (maybe p.d.f. in point Y=E[Y|X] has minimum, not maximum), and there are a lot of such subtle things.

Also let me remind two unsolvable problems in AI/ML, which can be for some reasons be forgotten, behind beauty slogans:

(1) It's interpolation methods, not extrapolation - it has no ability to deal with new problems

(2) nobody knows how any model will behave on data which is not from the same distribution (man in costume of banana for pedestrian localization)

• how about modeling the error from training data set to "predict" error for inference ? Jun 21, 2018 at 12:49
• Even assume it's additive "predict_for_mean" + "predict_for_error". You can imagine any schema to predict signal and error separately. But one more time - if we "only interpolate" we can not say something confidently. We predict temperature on the surface. Yes you can say this my prediction "20" and prediction for error is "5". So it say that I think that real response is lie in [20-5, 20+5] but to really understand what does it mean, we need to understand real phenomen and mathematical model. And ML is not about both of it. Other areas make some preliminary assumptions. Jun 22, 2018 at 9:26

I find that a simple method is MC dropout. In prediction you duplicate the case and expand that into a batch and enable the dropout, then you will obtain multiple outputs for the same input but with different dropped parameters. You get multiple outputs through one forward pass(and only one model) and then get a distribution of the output.