0
$\begingroup$

How to cross-validate a model with log-transformed x's and y's and obtain the RMSE in the original units? For example, I got an error message when I used the cvTools package:

 library(cvTools)
 library("robustbase")
 data("coleman")
 set.seed(1234)
 folds <- cvFolds(nrow(coleman), K = 5, R = 10)
 fitLm <- lm(log(Y) ~ ., data = coleman)
 repCV(fitLm, cost = rtmspe, folds = folds, trim = 0.1)
 Error in eval(expr, envir, enclos) : object 'Y' not found
$\endgroup$
1
$\begingroup$
library(cvTools)
library("robustbase")
data("coleman")
coleman<-transform(coleman,Y=log(Y))

set.seed(1234)
folds <- cvFolds(nrow(coleman), K = 5, R = 10)
fitLm <- lm(Y ~ ., data = coleman)
repCV(fitLm, cost = rtmspe, folds = folds, trim = 0.1)

5-fold CV results: CV 0.05513486

  • I only see Y transformation in your example, but you can apply any transformations beforehand.
| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you for your help. However, the results are the RMSE of log(Y). How to get the RMSE in the original units (is it EXP(0.05513486)?). Do I have to account for the retransformation bias? If yes, How? $\endgroup$ – KuJ Sep 29 '13 at 10:18
  • $\begingroup$ Yes, but it is not built in. You could take all predictions and true values which are generated, apply EXP() to undo transformation, and recalculated RMSE. $\endgroup$ – pat Oct 5 '13 at 23:01
  • $\begingroup$ Since you're using log transforms for both X and Y, consider reporting results in terms of percentage or fold change in Y per percentage or fold change of X, which comes directly from your analysis, and for which the standard errors will be symmetric about the predicted values. That might be easier for others to understand than back-transforming to the original scales, in which the back-transformed standard errors will necessarily be asymmetric and difficult to describe. $\endgroup$ – EdM Nov 27 '13 at 15:58
  • $\begingroup$ Unfortunately, exponentiating the predictions from the logged model is not sufficient. You will also need to multiply by the mean exponentiated residual to get back to the original scare. This will work as long as the errors are iid. Google "Duan (1983) smearing" for the paper. $\endgroup$ – Dimitriy V. Masterov Mar 1 '14 at 21:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.