3
$\begingroup$

A physics application I'm using reports for a first order fit of the three points below as $11.388612x - 301.878$.

   x, y    
  35, 0
  430, 4861
  656, 7000

It also shows a field labeled: "RMS: 329.499"

How is that RMS calculated? I tried RMSD as defined here but didn't get the same value.

$\endgroup$
3
  • $\begingroup$ Surprisingly, Wikipedia appears to have no articles related to this subject that explicitly and clearly show the correct formula in this least squares context! (The formula is buried in articles on analysis of variance and least squares.) $\endgroup$ – whuber Jan 13 '11 at 17:04
  • 2
    $\begingroup$ I think you are all confused, rms stands for your favorite GNU advocate, Richard Matthew Stallman $\endgroup$ – Chase Jan 13 '11 at 19:14
  • 1
    $\begingroup$ @Chase Good point; we shouldn't rely on acronyms: see stats.stackexchange.com/q/6039/919 . However, "Royal Meteorological Society" gets more Google hits than "Richard M Stallman" :-). $\endgroup$ – whuber Jan 13 '11 at 19:34
6
$\begingroup$

That's the root mean square error (RMSE) of the regression.
$$RMSE = \sqrt{\frac{1}{n-k}\sum{(y_i-\hat{y_i})^2}},$$
where $y_i$ is the observed and $\hat{y_i}$ the fitted value for observation $i$, $n$ is the number of observations, and $k$ is the number of parameters fitted (including the constant).

I just tried fitting a straight line by simple linear regression in another statistics package and got an RMSE of 329.499751.

$\endgroup$
6
  • $\begingroup$ Not correct: you need to use $n-2$, not $n$. $\endgroup$ – whuber Jan 13 '11 at 16:03
  • $\begingroup$ good point, fixed. $\endgroup$ – onestop Jan 13 '11 at 16:54
  • $\begingroup$ Can you confirm this thinking: with n = 3, if order is 2, then k is 3. So then n - k is zero, by which we don't want to divide. Is my understanding correct that if n = order + 1, there is always an exact fit, so calculating RMSE isn't even necessary? $\endgroup$ – Robert Frank Jan 14 '11 at 2:20
  • $\begingroup$ Yes. When n=k, the model becomes an exact fit to the data, i.e. the fitted values are the same as the observed values, so you'd get 0/0 in the above formula, so the result is undefined. But logically, as the 'errors' are all zero, you'd expect the root mean square error to be zero too, and that's what the stats packages i've tried report. $\endgroup$ – onestop Jan 14 '11 at 10:18
  • 1
    $\begingroup$ @Robert Frank: by asking this question you are effectively helping out anyone else who will in the future have the same doubt as you. :) $\endgroup$ – nico Jan 14 '11 at 18:11
8
$\begingroup$

RMS stands for the root mean square error. It's calculated in the following way.

  1. First we calculate the residuals: -96.72, 265.77, -169.05
  2. Next we calculate the squared residuals: -96.72$^2$, 265.77$^2$, -169.05$^2$
  3. Then we sum and divide by $n-2=1$
  4. Take the square root.

Further info

A residual is simply the $observed - fitted$. So when x = 35, the observed is 0 and the fitted value is

\begin{equation} 11.388612\times 35 - 301.878 = 96.72 \end{equation} The residual is then: $0 - 96.72 = -96.72$

$\endgroup$
1
  • $\begingroup$ Great answer, csgillespie. Thanks for taking the time to edit my question and provide your answer. The reason I didn't accept it was that in your step 3, I couldn't figure out where the constant "2" came from! I liked your presentation and clarity over the other answers, except for that. I did give it +1 though. Thanks for the help. $\endgroup$ – Robert Frank Jan 14 '11 at 13:31
4
$\begingroup$

It's the RMS (root mean square) of the residuals of the linear regression.

In R:

> x <-c(35, 430, 656)
> y <- c(0, 4861, 7000)
> mod <- lm(y~x)
> mod

Call:
lm(formula = y ~ x)

Coefficients:
(Intercept)            x  
    -301.88        11.39  

> sqrt(sum(resid(mod)^2))
[1] 329.4998
$\endgroup$
2
  • $\begingroup$ Thanks, Nico, for taking the time to answer. You assumed that a beginner like me knows what "R" is. I didn't ... until I just Googled it ... so your R code didn't help. $\endgroup$ – Robert Frank Jan 14 '11 at 14:28
  • 1
    $\begingroup$ Oh, OK! Well... if you did not find it, R is an open-source statistics environment and programming language. You can download it at r-project.org $\endgroup$ – nico Jan 14 '11 at 15:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.