2
$\begingroup$

Is taking the average of different rmse valid? for example average rmse = (rmse1+rmse2+rmse3)/3 Thank you for your help!

$\endgroup$
  • 3
    $\begingroup$ Valid for what exactly? Sure, you can average them you can multiply them, too. $\endgroup$ – gung - Reinstate Monica May 19 '14 at 14:13
  • 1
    $\begingroup$ if for example I want to know which is the best among different models and I get the rmse for three different input to each model. Therefore I have 3 rmse for each. Is it correct to just average the three rmse's in able to select the best model? $\endgroup$ – angelo May 19 '14 at 14:44
  • $\begingroup$ Its answer lies in reading sampling theory. $\endgroup$ – Subhash C. Davar May 19 '14 at 16:00
4
$\begingroup$

I actually wasn't sure about this either so I tested it out with a short example:

## Create simple function to calcualte the error
rmse <- function(error){sqrt(mean(error^2))}

## Define two example error vectors
error1 <- c(0.4, 0.2, 0.01)
error2 <- c(0.1, 0.3, 0.79)

## Find the RMSE of each error vector
rmse1 <- rmse(error1)
rmse2 <- rmse(error2)

## Compare the RMSE variants
print(rmse_all <- rmse(c(error1, error2)))
[1] 0.3924708
print(rmse_avg <- mean(rmse1, rmse2))
[1] 0.2582634

So we can se that they are not equal.

## As described by @whuber in the comments:

a <- rmse1^2*length(rmse1)                 # - square each rmse & multiply
b <- rmse2^2*length(rmse2)                 #   it by its associated count
c <- sum(a, b)                             # - sum that stuff up,                  
d <- c/sum(length(rmse1), length(rmse2))   # - divide by the total count,          
print(total_rmse <- sqrt(d))               # - take the square root.
[1] 0.3924708
$\endgroup$
  • 3
    $\begingroup$ The answer lies in front of you. Since you have defined rmse as the root of the mean of certain values, then just square each rmse, multiply it by its associated count, sum that stuff up, divide by the total count, and take the square root. The "slight bastardization" can be dreadfully wrong. $\endgroup$ – whuber Jan 14 '16 at 21:51
  • $\begingroup$ @whuber - I have edited to show what you describe. Please feel free edit my post with comments to add clarity. $\endgroup$ – n1k31t4 Jan 14 '16 at 22:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.