0
$\begingroup$

I have a dataset (X, y) where X is multi-dimensional features and y is the class label of each sample and it is a continues value between [-1,1]. I am using MLPRegressor as machine learning model to be used for prediction. To evaluate my model, I use several regression metrics found here, specifically sklearn.explained_variance_score, sklearn.r2_score, and sklearn.mean_squared_error. After training the model, I got the following results:

Variance_score: 0.98
R^2_score: 0.98
mean_squared_error: 0.02

I understand that for variance and r2 scores, the best value would be 1.0. However, I don't know what would be the best value of mean_squared_error. What does 0.02 tell? Is the smaller the better or the higher the better? Does the value always set in a certain range?

Thank you

$\endgroup$
2
$\begingroup$

0.02 can be a very good mean squared error. It can be so good that I might check for overfitting. You will understand better about how to interpret it once you understand how it is calculated. Mean squared error is defined as follows: Summation of squares of all (predicted - actual values) divided by the number of data points. You can see from the formula that the smaller it is the better. enter image description here

Here n is the number of data points, $\hat{Y}_i$ is the predicted value, $Y_i$ is the actual value.

You should divide you dataset into test and training sets and also perform cross validation to confirm that your model is not overfit. Also, there is no fixed range for the mean squared error. It depends on the size of your target variables.

$\endgroup$
  • $\begingroup$ Thank you for your response. I actually performed train_test_split but not cross-validation. I'll double check for possible overfitting. $\endgroup$ – Steven Oct 24 '17 at 23:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.