# What to look: R Square or Root Mean Squared Error (RMSE) for a multiple linear regression model?

I made a multiple linear regression model. This model uses 3 predictors to determine the response (Temperature in K). This model gives a R square of 0.234 (Indicates that this model is a failure). But when i calculated RMSE to be 0.935 K which is very good.

My model gives very good RMSE but very bad R square. So my question is, what conclusions can be made for this model in terms of its accuracy?

First, an $$R^2 = 0.234$$ isn't necessarily bad and an RMSE = $$0.935$$K isn't necessarily good. What is a good $$R^2$$ depends on the field of study and RMSE depends on the scale of measurement.

Second, the two measure different things. $$R^2$$ is a measure of how much of the variation in the DV is accounted for by the model. RMSE is a measure of how close the predicted values are to the actual ones. The code below gives some models with varying $$R^2$$ and RMSE. (Material after a # is result).

install.packages("Metrics")
library(Metrics)

set.seed(1234)

x1 <- rnorm(1000)
x2 <- rnorm(1000)
x3 <- rnorm(1000)

y1 <- 10*x1 + 10*x2 + 10*x3 + rnorm(1000,0, 5000)
y2 <- 10*x1 + 10*x2 + 10*x3 + rnorm(1000,0, 5)
y3 <- x1/10 + x2/10 + x3/10 + rnorm(1000,0, 5000)
y4 <- rnorm(1000,0,1)

m1 <- lm(y1~x1+x2+x3)
summary(m1)  #R2 = 0.000
rmse(m1$fitted.values, y) #59.9 m2 <- lm(y2~x1+x2+x3) summary(m2) #R2 = 0.93 rmse(m2$fitted.values, y) #20.11

m3 <- lm(y3~x1+x2+x3)
summary(m3)  #R2 = 0.000
rmse(m3$fitted.values, y) #2551 m4 <- lm(y4~x1+x2+x3) summary(m4) #R2 = 0.001 rmse(m4$fitted.values, y) #10.6