13
$\begingroup$

I have 3 datasets of the same system. But for the first one, I have 21 measurements. For the second and the third one I have only 9 measurements. Now I made a model using these 3 datasets (so 3 models, 1 per dataset). When I want to compare the error between these two datasets. Is there a clear advantage by using the MSE in stead of the LSE (least squared error). On the internet I do not find a clear answer for this. What are the main advantages?

$\endgroup$

3 Answers 3

16
$\begingroup$

I think you're confusing how to build a model from data and how to quantify a model accuracy once it's built.

When you want to build a model (linear regression in your case I guess?), you would usually use the least square error method that is minimizing the "total" euclidean distance between a line and the data points. Theoretically the coefficients of this line can be found using calculus but in practice, an algorithm will perform a gradient descent which is faster.

Once you have your model, you want to evaluate its performances. Thus, in the case of regression, it may be good to compute a metric which evaluate "how far" is your model to the actual data points (or test set data if you have one) in average. The MSE is a good estimate that you might want to use !

To sum up, keep in mind that LSE is a method that builds a model and MSE is a metric that evaluate your model's performances.

$\endgroup$
4
  • 1
    $\begingroup$ I made a non-linear biological inactivation model with 2 parameters with the matlab commando: LSQNONLIN. This commando gives me the least square error. I have 3 of these least squared errors because I did it for 3 datasets. Now I want to compare the accuracy of both datasets. Why can't i compare these LSE's to eachother? $\endgroup$
    – Thomas
    Apr 13, 2015 at 13:54
  • 1
    $\begingroup$ @Thomas Do the data sets have differing numbers of observations? Did you hold out a final data set to score all three models on? $\endgroup$ Feb 5, 2016 at 22:54
  • 2
    $\begingroup$ But the equation of LSE and MSE is almost the same, right? $\endgroup$
    – QtRoS
    Mar 5, 2018 at 10:55
  • 1
    $\begingroup$ Consider two data sets, one with 10 data points and one with 10,000 data points. If they have the same MSE they cannot have the same LSE - This is why the Mean in "Mean Squared Error" is used, the squaring makes all numbers positive and the mean averages those values so that the statistic is independent of the number of data points. R-squared (R2) calculated as "R2 = 1.0 - (absolute_error_variance / dependent_data_variance)" is also used for similar reason, that is, it is independent of the number of data points used in the modeling. $\endgroup$ Apr 9, 2018 at 11:26
8
$\begingroup$

MSE (Mean Squared Error) is mean of squared error i.e. the difference between the estimator and estimated. MMSE (Minumum Mean Square Error) is an estimator that minimizes MSE. Hence LSE and MMSE are comparable as both are estimators.LSE and MSE are not comparable as pointed by Anil. There are some important differences between MMSE and LSE, theoretically.
MMSE is optimal for all realizations of the process while LSE is optimal for the given data itself. This is because MMSE uses ensemble averages (expectation) while LSE uses time average.

What it means practically is : 1. For MMSE you need to know the second order statistical properties of the data (crosscorrelation and autocorrelation), while for LSE you need only the data. Autocorrelation & crosscorrelation is computationally expensive and an accurate calculation needs a lot of data points/experiments. 2. MMSE coefficients are optimal for the process so it is optimal for all datasets of the process while LSE is optimal only for the particular data set. LSE coefficients will not remain optimal if dataset changes.

Also please note that MMSE approaches LSE if the process is ergodic and the number of data points approaches infinity.

$\endgroup$
2
$\begingroup$

I believe the current first answer by Anil Narassiguin is misleading. It says at the bottom: "LSE is a method that builds a model and MSE is a metric that evaluate your model's performances."

This is simply not true. Basically, they are both loss/cost functions. Both calculate the error of the current predictions while iterating so the weights can be optimized.

However, LSE is used for classification issues while MSE is used for regression issues. I believe this is main difference between these two, so you need to figure out what kind of issue you have, regression of classification.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.