3
$\begingroup$

A physics application I'm using reports for a first order fit of the three points below as $11.388612x - 301.878$.

   x, y    
  35, 0
  430, 4861
  656, 7000

It also shows a field labeled: "RMS: 329.499"

How is that RMS calculated? I tried RMSD as defined here but didn't get the same value.

Improve this question
$\endgroup$
3
  • $\begingroup$ Surprisingly, Wikipedia appears to have no articles related to this subject that explicitly and clearly show the correct formula in this least squares context! (The formula is buried in articles on analysis of variance and least squares.) $\endgroup$
    – whuber ♦
    Jan 13, 2011 at 17:04
  • 2
    $\begingroup$ I think you are all confused, rms stands for your favorite GNU advocate, Richard Matthew Stallman $\endgroup$
    – Chase
    Jan 13, 2011 at 19:14
  • 1
    $\begingroup$ @Chase Good point; we shouldn't rely on acronyms: see stats.stackexchange.com/q/6039/919 . However, "Royal Meteorological Society" gets more Google hits than "Richard M Stallman" :-). $\endgroup$
    – whuber ♦
    Jan 13, 2011 at 19:34

3 Answers 3

Reset to default
6
$\begingroup$

That's the root mean square error (RMSE) of the regression.
$$RMSE = \sqrt{\frac{1}{n-k}\sum{(y_i-\hat{y_i})^2}},$$
where $y_i$ is the observed and $\hat{y_i}$ the fitted value for observation $i$, $n$ is the number of observations, and $k$ is the number of parameters fitted (including the constant).

I just tried fitting a straight line by simple linear regression in another statistics package and got an RMSE of 329.499751.

Improve this answer
$\endgroup$
6
  • $\begingroup$ Not correct: you need to use $n-2$, not $n$. $\endgroup$
    – whuber ♦
    Jan 13, 2011 at 16:03
  • $\begingroup$ good point, fixed. $\endgroup$
    – onestop
    Jan 13, 2011 at 16:54
  • $\begingroup$ Can you confirm this thinking: with n = 3, if order is 2, then k is 3. So then n - k is zero, by which we don't want to divide. Is my understanding correct that if n = order + 1, there is always an exact fit, so calculating RMSE isn't even necessary? $\endgroup$
    – Robert Frank
    Jan 14, 2011 at 2:20
  • $\begingroup$ Yes. When n=k, the model becomes an exact fit to the data, i.e. the fitted values are the same as the observed values, so you'd get 0/0 in the above formula, so the result is undefined. But logically, as the 'errors' are all zero, you'd expect the root mean square error to be zero too, and that's what the stats packages i've tried report. $\endgroup$
    – onestop
    Jan 14, 2011 at 10:18
  • 1
    $\begingroup$ @Robert Frank: by asking this question you are effectively helping out anyone else who will in the future have the same doubt as you. :) $\endgroup$
    – nico
    Jan 14, 2011 at 18:11
8
$\begingroup$

RMS stands for the root mean square error. It's calculated in the following way.

  1. First we calculate the residuals: -96.72, 265.77, -169.05
  2. Next we calculate the squared residuals: -96.72$^2$, 265.77$^2$, -169.05$^2$
  3. Then we sum and divide by $n-2=1$
  4. Take the square root.

Further info

A residual is simply the $observed - fitted$. So when x = 35, the observed is 0 and the fitted value is

\begin{equation} 11.388612\times 35 - 301.878 = 96.72 \end{equation} The residual is then: $0 - 96.72 = -96.72$

Improve this answer
$\endgroup$
1
  • $\begingroup$ Great answer, csgillespie. Thanks for taking the time to edit my question and provide your answer. The reason I didn't accept it was that in your step 3, I couldn't figure out where the constant "2" came from! I liked your presentation and clarity over the other answers, except for that. I did give it +1 though. Thanks for the help. $\endgroup$
    – Robert Frank
    Jan 14, 2011 at 13:31
4
$\begingroup$

It's the RMS (root mean square) of the residuals of the linear regression.

In R:

> x <-c(35, 430, 656)
> y <- c(0, 4861, 7000)
> mod <- lm(y~x)
> mod

Call:
lm(formula = y ~ x)

Coefficients:
(Intercept)            x  
    -301.88        11.39  

> sqrt(sum(resid(mod)^2))
[1] 329.4998
Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks, Nico, for taking the time to answer. You assumed that a beginner like me knows what "R" is. I didn't ... until I just Googled it ... so your R code didn't help. $\endgroup$
    – Robert Frank
    Jan 14, 2011 at 14:28
  • 1
    $\begingroup$ Oh, OK! Well... if you did not find it, R is an open-source statistics environment and programming language. You can download it at r-project.org $\endgroup$
    – nico
    Jan 14, 2011 at 15:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.