Mean squared error versus Least squared error, which one to compare datasets? I have 3 datasets of the same system. But for the first one, I have 21 measurements. For the second and the third one I have only 9 measurements. Now I made a model using these 3 datasets (so 3 models, 1 per dataset). When I want to compare the error between these two datasets. Is there a clear advantage by using the MSE in stead of the LSE (least squared error). On the internet I do not find a clear answer for this. What are the main advantages?
 A: MSE (Mean Squared Error) is mean of squared error i.e. the difference between the estimator and estimated. MMSE (Minumum Mean Square Error) is an estimator that minimizes MSE. Hence LSE and MMSE are comparable as both are estimators.LSE and MSE are not comparable as pointed by Anil. There are some important differences between MMSE and LSE, theoretically.
MMSE is optimal for all realizations of the process while LSE is optimal for the given data itself. This is because MMSE uses ensemble averages (expectation) while LSE uses time average.
What it means practically is : 
1. For MMSE you need to know the second order statistical properties of the data (crosscorrelation and autocorrelation), while for LSE you need only the data. Autocorrelation & crosscorrelation is computationally expensive and an accurate calculation needs a lot of data points/experiments. 
2. MMSE coefficients are optimal for the process so it is optimal for all datasets of the process while LSE is optimal only for the particular data set. LSE coefficients will not remain optimal if dataset changes. 
Also please note that MMSE approaches LSE if the process is ergodic and the number of data points approaches infinity. 
A: I think you're confusing how to build a model from data and how to quantify a model accuracy once it's built.
When you want to build a model (linear regression in your case I guess?), you would usually use the least square error method that is minimizing the "total" euclidean distance between a line and the data points. Theoretically the coefficients of this line can be found using calculus but in practice, an algorithm will perform a gradient descent which is faster.
Once you have your model, you want to evaluate its performances. Thus, in the case of regression, it may be good to compute a metric which evaluate "how far" is your model to the actual data points (or test set data if you have one) in average. The MSE is a good estimate that you might want to use !
To sum up, keep in mind that LSE is a method that builds a model and MSE is a metric that evaluate your model's performances. 
A: I believe the current first answer by Anil Narassiguin is misleading. It says at the bottom: 
"LSE is a method that builds a model and MSE is a metric that evaluate your model's performances."
This is simply not true. Basically, they are both loss/cost functions. Both calculate the error of the current predictions while iterating so the weights can be optimized.
However, LSE is used for classification issues while MSE is used for regression issues. I believe this is main difference between these two, so you need to figure out what kind of issue you have, regression of classification.
