What is the relationship between mean squared error and classification error?

I've trained a network using a genetic algorithm and I have two possible fitness functions for my GA: MSE and CErr.

If I use MSE as my fitness function, over time MSE decreases and classification error decreases. Looking at my plots, it looks like MSE and CErr are directly related. However, if I use CErr as my fitness function, over time my CErr decreases but my MSE doesn't always decrease.

For example, consider these two training attempts, the first with MSE as fitness function and second with CErr as fitness function. The data, train/test split, activation function, etc everything is equal except fitness function. Accuracy is just a function of classification error (1 - CErr = Acc).

From this I can deduce a lower MSE means a lower CErr but a lower CErr doesn't always mean a lower MSE. For a genetic algorithm, I should use MSE as my fitness function because if MSE were to rise to reach a better CErr, it might get trapped in local minima as better solutions are guaranteed when the MSE is lower (and the algorithm, through crossover and mutation, will focus on areas in the search volume which are suboptimal).

But why do I get results like this in the first place? I thought MSE was directly proportional to CErr and vice-versa but I do not always observe this.

• Interesting. My network does consider, in the case of a binary classification problem, a output of 0.51 for one class as belonging to that class. I guess even if CErr is optimized, CErr is not a suitable measure because it doesn't show confidence in classification. If an CErr optimized network only cared about cutoffs of probabilities, an MSE optimized network might not and my neural outputs might be closer to 1.00 than the CErr optimized network, thus being more confident in classification. Apr 18, 2020 at 0:34
• To make what I said short, my neural outputs in the CErr optimized network might be within the range [0.51, 1.00] whereas the MSE optimized network might be [0.90, 1.00] to give an example. Apr 18, 2020 at 0:35