4
$\begingroup$

When using neural network for classification problem, and using softmax as last layer for last layer.

Typically, we have a prediction and a confidence level. However, is there such confidence interval measure for neural network regression problem?

$\endgroup$
2
$\begingroup$

You would have to output vectors of means and standard deviations rather than discrete values to achieve that.

One solution to get those vectors would be variational inference - generate those, sample w/reparametrization, then optimize so the results of the sampling match the original values like in normal regression (i.e. MSE/MAPE/MAE/whatever loss) and regularize the means and stddev to 0/1 respectively.

Essentially the same process as a vanilla Variational Autoencoder, except you're not bound by the Autoencoder architecture, and you want the means/stddevs as the outputs of the trained network rather than the sampled values.

$\endgroup$
  • $\begingroup$ Could you elaborate a bit more on how you would get the network to output the means and standard deviations alongside the actual predicted target value? $\endgroup$ – Aesir Dec 13 '18 at 7:48
  • $\begingroup$ You just use two neurons per each latent space dimension, one for the mean, one for stddev, and your final 'concrete' output is a sample drawn from the resulting distribution (possibly processed by further hidden layers). Just by having a sampling layer consistently treating the two outputs as mean/stdev and backprop, the outputs wind up effectively being those things. $\endgroup$ – jkm Jan 7 at 11:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.