# Can a neural network output represent a posterior probability?

I seem to remember from years ago when I first read Bishop's ANN book that it is possible to construct a neural network such that the outputs should represent the posterior probability that I would have found if I used the likelihood approach to separate the same data. Is this true or have I made this up in my mind? If it is true what are the conditions, I seem to remember it has to have no loops and limits on the number of hidden layers. Although I also seem to remember a result from Komogorov showing that many hidden layers aren't helpful if the network is well trained. Unfortunately I can't find the book anymore and I can't seem to find much on this result which makes me worried it's me.

I think I probably need to flesh out my question a bit as pointed out by @bayerj

I have a few variables (I'll assume I only have one for now) which I am using to find the probability of one of three hypotheses being true given the three variables. I have some data which which a can construct PDFs. I am using

$$P(H_{k}|x) = \frac{P(x|H_{k})P(H_k)}{P({x})}$$

where P(x) is the unconditional probablity density

$$P(x) = \sum P(X|H_k)P(H_k)$$

I'm trying to verify whether it is possible to train a network, using the data with which I construct my PDFs, such that the value of the output node equals P(H_k|x). I'm sure this is possible I just can't remember the working. Since I posted the question I have found http://www-vis.lbl.gov/~romano/mlgroup/papers/neural-networks-survey.pdf which I think is saying what I want to do is possible.

• What do you mean by posterior probability, likelihood approach exactly? Can you be more specific with your question in the problem statement? Commented Oct 24, 2012 at 9:50
• Certainly easy to make a virtual machine representation of Bayesian analysis given the prior is proper so that the possible unknowns (e.g. parameters) can be sampled and then the possible knowns (e.g. data) can be sampled given the unknowns (e.g. parameters) sampled. Stigler points out that it is very general albiet often computationally not feasible yet. Fig 5 Stigler 2010 here stats.stackexchange.com/questions/39177/… Commented Oct 24, 2012 at 12:43