Why the MSE function has the square? There is mse function: C = $\frac{1}{2n}$ * $\sum(length(y - a)^2)$
why not just use C = $\sum(length(y - a))$ ?
(where "length" is the vector's length, "y" - ideal network's output, "a" - current network output)
 A: You're talking about L1 norm and L2 norm. Both work for neural networks. However, they are different:


*

*L1 norm is better for sparsity and robust against outliers 

*L2 norm is more sensitive to large errors (square those large errors)

*Their first derivative is very different. I don't want to repeat what someone has already written. Look at https://stats.stackexchange.com/a/159379/34623.

*How those error functions update your weight is different (gradient). This has  significant impact on your convergence in stochastic gradient decent (or something like that).

*http://www.chioka.in/differences-between-the-l1-norm-and-the-l2-norm-least-absolute-deviations-and-least-squares/
Without more information, I can't comment on how L2 norm is better (or worse) for your problem.
A: Short answer: both can be used.  
Longer answer: both measures are in active use. The first measure is based on the Euclidean distance, the second one on the taxi-cab distance. Or more formally: the $L_2$ distance and the $L_1$ distance.
Which is better depends on the context. Intuitively: the Euclidean distance prefers many small/medium errors over a few big errors while the taxi-cab distance is more forgiving when it comes to a few large errors. Which one is preferable depends on the context and what you are trying to achieve.
