How to evaluate neural network regression model

Question

I have some data with 2963 observations and 7 variables. I want to use regression and train this data using neural network then evaluate the regression model.

I've tried splitting the data into training 70% and testing 30% then train the data using neuralnetwork library in R.

I've tried evaluating the model using MSE and SSE for MSE I got a value of 0.01026 for SSE I got a value of 4.56215

I don't understand the difference between these two measures and I can't tell if the values I got are good or not. Please help!

Answer 1

There are multiple loss functions you can use:

• MSE, aka Mean Squared Error: take all the errors, square them, and find the mean.
• RMSE, aka Root Mean Squared Error: Squared root of MSE.
• SSE, aka Sum of Squared Errors: take all the errors, square them, and compute their sum.

What your MSE value is telling you is that the square of the errors are, one average, 0.01026 units away from the true (test) values.

What the SSE is telling you instead is that the sum of the squares of all your errors (it's like a 'total amount of inaccuracy').

If you find these interpretations troublesome you can take the RMSE, which tells you how distant your predictions are, on average, from the true (test) values. This is, in my opinion, better than MSE, since RMSE is a mean computed on the same scale of your dependent variable.

Whether the values are good or not, well that's not something you can infer from those coefficients alone. These scores make more sense when you compare different models. In that case, by looking at the error coefficients, you can determine whether a model is better than another in explainins the same dependent variable.