maximum a posteriori vs squared loss I am unclear about max a posteriori and squared loss.
Let me assume I have $N$ images and $\mathbf{y}_i$ is the label of the image $i$, where,  $\mathbf{y}_i\in \mathbb{R}^{C\times 1}$ - a binary vector of lenght $C$ (number of classes). $\mathbf{y}_{ic} = 1$ when the image $i$ belonging to class $c$.
Consider the following two loss functions.
\begin{align}
          L_1 &= -\sum_i\sum_{c=1}^C y_{ic}\, \log(P(y_{ic}|\mathcal{D})) \\
     L_2 &= \sum_i \sum_{c=1}^C \left(y_{ic}-P(y_{ic}|\mathcal{D})\right)^2
\end{align}
What is the difference between these functions? When should I use $L_1$ and when should I use $L_2$? 
I observed that in neural networks always $L_2$ is used. Why?
 A: L2 is not always used with neural networks, indeed for statistical pattern recognition problems the cross-entropy loss (with a softmax activation function for the output layer) is the preferred option.  See chapter 6 of Bishop's excellent book on Neural Networks for Pattern Recognition.
Secondly, I would say that L1 would be a maximum likelihood approach, rather than maximum a-posteriori as there is no prior distribution involved, just a likelihood.
Asymptotically (in the limit of an infinite amount of data and hidden units), both L1 and L2 will give the same answer, because the minimiser of L1 is given when the output of the model is the true probability of class membership, and the minimiser of L2 is the conditional mean of the target variable, which in this case is also the true probability of class membership.
The difference arises away from these asymptotic conditions (i.e. more or less every practical case), where I would suggest that L1 is probably more efficient in converging to the optimal solution in terms of the number of training samples given, but I am not confident that the practical difference is likely to be great for most problems.
In practice, most reasonable loss functions are suitable for training neural networks (see Saerens et al), however I almost always use a maximum likelihood criterion for training neural networks because this is the most theoretically sound approach as it most accurately describes the variability of the target variable around its conditional mean.
