In the case of a clasification feed-forward neural network, given that your output activation is a sigmoid $[0,1]$ then you actually have a binomial distribution. If you have a single output neuron you get a Bernoulli distribution is a special case of the binomial distribution with $n=1$. (https://en.wikipedia.org/wiki/Bernoulli_distribution)
Now, as you can see in wiki the variance of a prediction of a Bernoulli distribution is given by $var = p(1-p)$, so therefore you can say that the higher prediction the less variance you have, hence, the more confident you are. There is on-going research in the field as this is not a good estimate of confidence.
In case of continuous data, you could follow another approach that has been recently introduced to the field of Bayesian Optimisation, and fits your case (http://arxiv.org/abs/1502.05700). What the authors suggest is to train your network using the mean squared error of a linear output layer (after your non-linear activation tanh or sigmoid), and then train a bayesian linear regression model. With this way, you can have proper bayesian confidence intervals and it is empirically proven that they work.
More specifically, the implementation in Torch7 would be:
-- DNGO network
model = nn.Sequential()
Finally, the authors fit a Bayesian Linear regression to the predicted outputs against the target values, obtaining the confidence intervals. I believe this would be a very good fit for your case as well by just altering the network architecture.