# Mean field theory and neural networks

Mean field algorithm has been proposed to be used in combination with convolutional networks and recursive neural networks. What is the purpose of doing this? Is the goal to estimate a probability distribution which can be used for the loss function (to perform maximum likelihood)?

• Can you provide more details about where this was proposed and what the context was? – MachineEpsilon Nov 2 '17 at 5:04
• Here is a paper: arxiv.org/pdf/1606.07230.pdf references 24 and 26 propose some similar techniques. – Cauchy Nov 2 '17 at 17:18

## 1 Answer

The use of probablistic models is a common way to formulate the problem of semantic segmentation, but the interest is in assignment of labels to pixels not the uncertainty associated with the labels. Markov random fields and recurrent neural networks are commonly used as a refinement step to improve performance.

In semantic segmentation you want to assign discrete label $w \in \{1 . . . M\}$ to each pixel so that we know which of the $M$ objects is present, and we want to do this based on image data $I$. Now the adjacency of pixels in $I$ has a grid structure, so it's natural to want to encourage smoothness on the $w_i$. A classical way to do this in computer vision is by imposing a grid-based graphical model such as pairwise Markov random field model as a prior on the pixels.

$$P(w) = \frac{1}{Z} \Pi^J_{j=1} \phi_j(w_{C_j})$$ where Z is the normalisation constant, $\phi_j$ is a potential function and $C_j$ is a clique in the graph. Inference in these types of models is okay as long potentials are convex and the cliques are small. Unfortunately, non-convex potentials and large cliques are natural things to want because they better model real images.

The work that you mention in [26], is based on the paper in [18] in which they define an approximate inference algorithm for a full connected CRF (another type of graphical model). In particular, in [26] they start with fully connected CNN (a popular way to do semantic segmentation) which predicts pixel labels without regard for structure and then the refine the predicted pixels with a CRF. The trick is that they show that the mean field algorithm for training the CRF can be approximated by a sequence of CNN layers, and thus that CRF model can be approximated by RNN. This lets you train the CRF part and the fully-connected CNN part at the same time.