7
$\begingroup$

It is a common practice to measure an SVM model's performance by calculating its MSE (Mean Square Error). Why not use Mean Absolute Error (averaging errors' absolute values instead of squared values)?

$\endgroup$
1
  • $\begingroup$ Yan, I've merged your two accounts. Please, register this one to benefit from Stack Exchange facilities, such as voting, system notifications, commenting, etc. (BTW, it would be good to review your past questions and check whether any of their associated replies deserve some votes or green check--i.e., if one answer solved your problem, mark it as such so that future users will know that it was helpful.) $\endgroup$
    – chl
    Jul 26, 2012 at 13:13

3 Answers 3

15
$\begingroup$

Actually, looking at both MAE and RMSE gives you additional information about the distribution of the errors:

$\mathrm{MAE} \leq \mathrm{RMSE} \leq \mathrm{MAE}^2$ (for regression)

  • if $\mathrm{RMSE}$ is close to $\mathrm{MAE}$, the model makes many relatively small errors
  • if $\mathrm{RMSE}$ is close to $\mathrm{MAE}^2$, the model makes few but large errors
$\endgroup$
7
$\begingroup$

The choice of performance metric depends on what is important for the application that you are interested in. The MSE is a good performance metric for many applications as there is good reason to suppose that noise process is Gaussian. Sometimes it is better to use the MAE if you don't want your performance metric to be overly sensitive to outliers. Essentially there is no correct performance metric without knowing more about the nature of the application.

On a different note I am not overly keen on support vector regression, as quite often there is a knowledge about the distribution of the noise in the response variable and we are likely to get a better model of the data if we build that expert knowledge into out model. That is why we have GLMs rather than just using least squares regression for everything. The loss function used in SVM does not have a very clear statistical interpretation of this nature. SVM regression also uses a loss function that is based on a sort of worst-case bound on the error, so if you use a performance metric that is essentially an average case statistic, then that suggests you should instead use a model based on average case performance rather than worst case (e.g. a GLM).

$\endgroup$
1
$\begingroup$

MAE is more intuitive than MSE to simply evaluate the overall error.

MSE is easier to handle mathematically for variance analysis. For example, MSE is used to calculate the error variance $s_e^2$, which is a recurring value in regression statistics.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.