I am completely new to ML and R and I just want to understand why my Residual Standard error went down when i log replace my dependant variable with log(y). I am running regression using R lm
Initial formula: y~ time(x1) + x2 + x3 This gave RSE : 60.37
I replaced the formula with: log(y) ~ time(x1) + x2 + x3 This gave RSE: 0.56
Please let me know what I am missing!
The main reason is that you can not compare the residuals of the model y ~ ... with the residuals from the model log(y) ~ .... The residuals of your model log(y) ~ ... are the differences
log(y) - fitted.values(lm(log(y) ~ ...))
Here an example to illustrate the issue:
set.seed(42)
x <- 1:20
y <- runif(20, 4, 10)
m1 <- lm(y ~ x)
summary(m1)
m2 <- lm(log(y) ~ x)
summary(m2)
fitted.values(m1)
fitted.values(m2)
exp(fitted.values(m2))
You have to retransform the fitted values from the model log(y) ~ ... for comparability, see the exp(fitted.values(m2)) and compare this to the other fitted values.
Residual Standard Error defines the standard deviation(σ) of the residuals in OLS i.e. errors are assumed to be normally distributed with mean 0 and standard deviation σ. Lesser this value better the results(obviously other assumptions of regressions should also hold true)
Removed some text..as we cannot compare the 2 residuals
The residual standard error (RSE) is expressed on the same scale as your dependent variable y. So you could compare it to the standard deviation (SD) of the dependent variable y. In general, if the model does a good job at explaining the variability in the values of y, RSE would be expected to be lower than SD. In your case:
You can compare RSE from model y ~ time(x1) + x2 + x3 with the SD of y;
You can compare RSE from model log(y) ~ time(x1) + x2 + x3 with the SD of log(y).
It's not meaningful to compare the RSE from model y ~ time(x1) + x2 + x3 with the RSE from the model log(y) ~ time(x1) + x2 + x3, since they are expressed on different scales.
