How to determine the confidence of a neural network prediction?

Question

To illustrate my question, suppose that I have a training set where the input has a degree of noise but the output does not, for example;

# Training data
[1.02, 1.95, 2.01, 3.06] : [1.0]
[2.03, 4.11, 5.92, 8.00] : [2.0]
[10.01, 11.02, 11.96, 12.04] : [1.0]
[2.99, 6.06, 9.01, 12.10] : [3.0]

here the output is the gradient of the input array if it were noiseless (not the actual gradient).

After training the network, the output should look something like this for a given input.

# Expected Output
[1.01, 1.96, 2.00, 3.06] : 95% confidence interval of [0.97, 1.03]
[2.03, 4.11, 3.89, 3.51] : 95% confidence interval of [2.30, 4.12]

My question is how can a neural network be created such that it will return a predicted value and a measure of confidence, such as a variance or confidence interval?

Answer 1

It sounds like you are looking for a prediction-interval, i.e., an interval that contains a prespecified percentage of future realizations. (Look at the tag wikis for prediction-interval and confidence-interval for the difference.)

Your best bet is likely to work directly with NN architectures that do not output single point predictions, but entire predictive distributions. You can then directly extract desired prediction intervals (or mean, or median point predictions) from these distributions. I and others have been arguing that predictive distributions are much more useful than point predictions, but to be honest, I have not yet seen a lot of work on predictive distributions with neural nets, although I have been keeping my eyes open. This paper sounds like it might be useful. You might want to search a bit, perhaps also using other keywords like "forecast distributions" or "predictive densities" and such.

That said, you might want to look into Michael Feindt's NeuroBayes algorithm, which uses a Bayesian approach to forecast predictive densities.

Answer 2

I'm not sure you can compute a confidence interval for a single prediction, but you can indeed compute a confidence interval for error rate of the whole dataset (you can generalize for accuracy and whatever other measure you are assessing).

If $e$ is your error rate while classifying some data $S$ of size $n$, a 95% confidence interval for your error rate is given by: $$ e \pm 1.96\sqrt{\frac{e\,(1-e)}{n}}$$.

(see "Machine Learning" book from Tom Mitchell, chapter 5.)

EDIT

Guess I should state a more general case, which is: $$ e \pm z_N\sqrt{\frac{e\,(1-e)}{n}},$$ where common choices for $z_N$ are listed in the following table:

confidence level    80%    90%    95%    98%    99%
values of zN       1.28   1.64   1.96   2.33   2.58

Answer 3

In terms of directly outputting prediction intervals, there's a 2011 paper 'Comprehensive Review of Neural Network-Based Prediction Intervals'

They compare four approaches:

1: Delta method 2: Bayesian method 3: Mean variance estimation 4: Bootstrap

The same authors went on to develop Lower Upper Bound Estimation Method for Construction of Neural Network-Based Prediction Intervals which directly outputs a lower and upper bound from the NN. Unfortunately it does not work with backprop, but recent work made this possible, High-Quality Prediction Intervals for Deep Learning.

Alternative to directly outputting prediction intervals, Bayesian neural networks (BNNs) model uncertainty in a NN's parameters, and hence capture uncertainty at the output. This is hard to do, but popular methods include running MC dropout at prediction time, or ensembling.

Answer 4

I don't know of any method to do that in an exact way.

A work-around could be to assume that you have gaussian noise and make the Neural Network predict a mean $\mu$ and variance $\sigma$. For the cost function you can use the NLPD (negative log probability density). For datapoint $(x_i,y_i)$ that will be $-\log N(y_i-\mu(x_i),\sigma(x_i))$. This will make your $\mu(x_i)$ try to predict your $y_i$ and your $\sigma(x_i)$ be smaller when you have more confidence and bigger when you have less.

To check how good are your assumptions for the validation data you may want to look at $\frac{y_i-\mu(x_i)}{\sigma(x_i)}$ to see if they roughly follow a $N(0,1)$. On test data you again want to maximize the probability of your test data so you can use NLPD metric again.

Answer 5

There are actually ways of doing this using dropout. Run the evaluation with dropout enabled (it's usually disabled for evaluation but turned on when training), and run the evaluation several times.

The result distribution from multiple different runs can be used as confidence intervals.

See the paper "Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning" Watch the youtube presentation Andrew Rowan - Bayesian Deep Learning with Edward (and a trick using Dropout)

Answer 6

Prediction intervals (PI) in non parametric regression & classification problems, such as neural nets, SVMs, random forests, etc. are difficult to construct. I'd love to hear other opinions on this.

However, as far as I know, Conformal Prediction (CP) is the only principled method for building calibrated PI for prediction in nonparametric regression and classification problems. For a tutorial on CP, see Shfer & Vovk (2008), J. Machine Learning Research 9, 371-421 [pdf]

Answer 7

I have not heard of any method that gives a confidence interval for a neural network prediction. Despite a lack of formal methodology, it seems like it might be feasible to construct one. I have never attempted this due to the compute power that would be needed and I make no claims on this working for certain, but one method that might work for a tiny neural net (or with blazing fast GPU power it could work for moderate sized nets) would be to resample the training set and build many similar networks (say 10,000 times) with the same parameters and initial settings, and build confidence intervals based on the predictions for each of your bootstrapped net.

For example, in the 10,000 networks trained as discussed above, one might get 2.0 (after rounding the neural net regression predictions) 9,000 of those times, so you would predict 2.0 with a 90% CI. You could then build an array of CIs for each prediction made and choose the mode to report as the primary CI.

Answer 8

This article is an interesting presentation by Yarin Gal of a technique based on dropout:

http://mlg.eng.cam.ac.uk/yarin/blog_3d801aa532c1ce.html

Archived version: https://web.archive.org/web/20210422213844/http://mlg.eng.cam.ac.uk/yarin/blog_3d801aa532c1ce.html

Answer 9

Work in Progress...

There has been a lot of work on predictive intervals for neural nets going back over the years:

The simplest approach (Nix and Weigend, 1994) is to train a second neural network to predict the mean-squared error of the first. Regression networks trained to minimise the mean-squared error learn the conditional mean of the target distribution, so the output of the first network is an estimate of the conditional mean of the targets and the second learns the conditional mean of the squared distance of the targets from the mean, i.e. the conditional variance. In practice, they don't have to be separate networks, you can have one network with two outputs, one for the conditional mean and one for the conditional variance.

References

D. A. Nix and A. S. Weigend, "Estimating the mean and variance of the target probability distribution," Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94), 1994, pp. 55-60 vol.1, doi: 10.1109/ICNN.1994.374138.

David A. Nix, Andreas S. Weigend, Learning Local Error Bars for Nonlinear Regression, NIPS 1994 (pdf)

CM Bishop, CS Qazaz, Regression with input-dependent noise: A Bayesian treatment, Advances in neural information processing systems, 347-353, 1997 (pdf)

Answer 10

I find that a simple method is MC dropout. In prediction you duplicate the case and expand that into a batch and enable the dropout, then you will obtain multiple outputs for the same input but with different dropped parameters. You get multiple outputs through one forward pass(and only one model) and then get a distribution of the output.

Answer 11

There is no way, all ML models is not about phenomen understanding, it's interpolation methods with hope "that it works". Start with such questions confidence, robustnes to noise there is no answers.

So to derive something please use various applied and fundamental science:

• Use control (and make assumption about dynamics)

• Use convex optimization (with some extra condition on function)

• Use math statistics (with preliminary assumptions on distributions)

• Use signal processing (with some assumptions that signal is band limited)

Scientist use some prelimiary assumptions (called axioms) to derive something.

There is no way to give any confidence without some preliminary assumption, so problem in not in DL mehtod, but it's problem in any method which try to interpolate without ANY preliminary assumption-there is no way to derive via algebra something intellegently without an assumption.

NN and various ML methods are for fast prototyping to create "something" which seems works "someway" checked with cross-validation.

Even more deeper the regression fitting E[Y|X] or it's estimate can be absolutely incorrect problem to solve (maybe p.d.f. in point Y=E[Y|X] has minimum, not maximum), and there are a lot of such subtle things.

Also let me remind two unsolvable problems in AI/ML, which can be for some reasons be forgotten, behind beauty slogans:

(1) It's interpolation methods, not extrapolation - it has no ability to deal with new problems

(2) nobody knows how any model will behave on data which is not from the same distribution (man in costume of banana for pedestrian localization)