MSE formula in Neural Network applications In Neural Network examples that I have seen online - sometimes the Mean Square Error is presented as
$$ 
MSE =  \frac{{1}}{2n}  \sum_{i}^{n} ( \widehat{y_i} -y_i)^2 \quad (1)
$$
and other times
$$ 
MSE =  \frac{{1}}{2}  \sum_{i}^{n} ( \widehat{y_i} -y_i)^2 \quad (2)
$$
Where I guess $n$ is the number of output nodes. 
Which one is the correct formula to use? Are we minimizing the total error or the "average" error between each output as in linear regression?
Also, bonus question: Do we really need to multiply by $1/2$? In my opinion, it is not that much more convenient than multiplying by $2$ when we take the derivative. 
Thank you
Update based on feedback below
$$ 
MSE =  \frac{{1}}{n}  \sum_{i}^{n} (y_i - \widehat{y_i})^2 \quad (3)
$$
Where $y_i$ is the desired Neural Network output, and $\widehat{y_i}$ is the neural network output.
 A: It's perfectly straightforward, I think:
To get mean square error, you take the errors ($y_i-\hat{y}_i$ -- and no, the error is not $\hat{y}_i-y_i$), you square them and then take their mean. Mean-square-error, just like it says on the label.
So, correctly, $MSE =  \frac{{1}}{n}  \sum_{i}^{n} (  y_i-\hat{y_i})^2$
(Anything else will be some other object)
If you don't divide by $n$, it can't really  be called a mean; without $\frac{1}{n}$, that's a sum not a mean. 
The additional factor of $\frac12$ means that it isn't MSE either, but half of MSE.
That said, you can halve it or double it, multiply by $n$ and so on without changing the argmin so if that's all you're doing that won't matter -- but a statistician does more with MSE than merely minimize it, so to me it matters to call things by their right names.
If - and here I address the authors you're reading, not you the innocent asker - you want to halve MSE, call it half-MSE. If you calculate SSE, say so. Don't play silly games with perfectly descriptive terms because you can't be bothered to type 5 extra characters. It's not hard!
A: What matters is the sum of squares. The rest is not important mathematically for optimization of a finite sample. Obviously, for an infinite size sample the sum is infinite, and things don't work out so well.
A: You're seeing both equations because they're used at different points in training. 
$$\frac{{1}}{2}  \sum_{i}^{n} ( y_i - \widehat{y_i})^2 \quad$$
is the error of a single training example, where n is the number of output nodes. The second equation, 
$$\frac{{1}}{2n}  \sum_{i}^{n} ( y_i - \widehat{y_i})^2 \quad$$
is the error of the network across all n training examples, though these are usually broken up into mini-batches to make calculation less expensive.
Also, $(\widehat{y_i} -y_i)^2 = (y_i - \widehat{y_i})^2$.
Source: http://neuralnetworksanddeeplearning.com/chap2.html 
Old question but still the top Google result for "MSE neural network"
A: Each sample in your minibatch gives you a squared error $(y_i-\hat{y_i})^2$. $MSE$ is the mean of this value for all $n$ samples in your minibatch:
$$MSE=\frac{1}{n}\sum_{i=1}^{n}(\hat{y_i}-y_i)^2$$
Notes


*

*There is no $\frac{1}{2}$ factor in $MSE$.

*Both $MSE$ and $(\hat{y_i}-y_i)^2$ are scalars.

A: The factor of two is used to cancel the exponent 2 at every differentiation. Otherwise, MSE is defined as in the answer by Soroush.
A: I see that: because there are three types of layers (input, Hidden, and output)
then dividing by (2) is correct, that's the number (n) is belongs the number of layers minuses one
