Calculating Neural Network Error

Question

I am confused with these two error formulas for artificial neural networks: \begin{align} \text{Error} &= {\rm target} - {\rm output} \\[7pt] \text{Mean Square Error} &= \frac{1}{2m} \sum_{i=1}^{m} (y-\hat y)^2 \end{align}

Training error and testing error are used to determine the progress of training, but I am wandering which error formula to use.

When we talk about "error" in a neural network, what does this "error" typically refer to?

//AForge
var teacher = new BackPropagationLearning(network);
//...
var error = teacher.RunEpoch(input, output);

Answer 1

If you're referring about evaluating the performance of a model you might find this useful.

Regardless of the model, the performance is evaluated by how well it can generalize. Meaning, how well it performs on the test data. So when you're describing the performance of your model to someone new to the context, what you tell him is how well your model did on the testing set-he doesn't care how well you've performed on the training set.

About the cross validation - it is the data set that we use to evaluate the model during our process of improving the model. This training set is used to identify what are problems the model is having; underfitting, overfitting. We have to go back and forth between training and cross validation to get an optimum performance. If we have a big enough data set, it's good practice to split the data in to training set(about 60%), CV set(about 20%) and test set(about the rest 20%). We never touch the test data until the last part of the experiment.

About the equations you've given, they're standard squared error formulas. Each of the equations only have the training, cv and test data as their domains respectively.