4
$\begingroup$

Does anyone know where this metric came from ?

Can someone bring article references or something like this?

Im actually wondering if there's any mathematical concept or any way to demonstrate mathematically that this metric is consequence from any model.

Improve this question
$\endgroup$
1

2 Answers 2

Reset to default
6
$\begingroup$

RMSE arises from what is probably the most important model in statistics, linear regression. A linear regression model is fit with least squares, which means minimizing the mean square error (MSE) for the sample. Take the square root of MSE, so that it's on the same scale as the data and hence easier to interpret, and you get RMSE.

If that just makes you wonder why we fit linear regression models with least squares, here's how Wikipedia puts it:

The OLS [ordinary least squares] estimator is consistent when the regressors are exogenous and there is no perfect multicollinearity, and optimal in the class of linear unbiased estimators when the errors are homoscedastic and serially uncorrelated. Under these conditions, the method of OLS provides minimum-variance mean-unbiased estimation when the errors have finite variances. Under the additional assumption that the errors be normally distributed, OLS is the maximum likelihood estimator.

That's a lot of nice properties.

Improve this answer
$\endgroup$
2
  • $\begingroup$ That's sound great. I do understand that linear regression can be achieved with least squares residual. But why we want to minimize the the squared residual?where this idea came from? Can we prove the meaningful of Squared residual or the MSE it self ? $\endgroup$
    – Alvaro Silvino
    Commented Jul 2, 2016 at 16:49
  • $\begingroup$ The Wikipedia quote above describes the motivation for minimizing MSE. Do you find it unclear? $\endgroup$
    – Kodiologist
    Commented Jul 2, 2016 at 16:51
4
$\begingroup$

The method of least squares was first developed by French mathematician Adrien-Marie Legendre and published in 1806.

The broad question at the time was how to combine multiple observations of celestial bodies to estimate the parameters that governed their motion.

  • More observations than parameters creates an overdetermined system of equations.
  • An overdetermined system of equations generally cannot be solved. What instead should one do?
  • Early work averaged observations together until there were as many observations as parameters, creating a solveable linear system. Legendre instead proposed minimizing the sum of square error.

Later, Gauss connected the method of least squares to his work on the normal distribution, showing that least squares is in a sense optimal with a linear equation and normally distributed errors.

For the full history, see Stephen Stigler's, The History of Statistics: Measurement of Uncertainty before 1900

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks for The references that's what I really need to read. Thanks !! $\endgroup$
    – Alvaro Silvino
    Commented Jul 2, 2016 at 17:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.