Using the trainControl and train functions from the caret package in R, I evaluated several different regressions (linear, log-log, exponential, quadratic, and logistic) using the leave-one-out cross-validation (LOOCV). Each regression is a model of input use (fertilizer or pesticides use) or input emissions (synthetic fertilizer emissions or fuel emissions on-farm) as a function of crop yields with a fixed effect for the country. An example model for fertilizer use is:

log(fertilizer) ~ log(yield) + factor(country)

The LOOCV in R using the train and trainControl functions report MAE, RMSE, and R^2 values. Should I use MAE or RMSE to evaluate the regressions? Should results for the exponential, log-log, and polynomial regressions be transformed (so should I take the square root of the MAE or RMSE errors for the quadratic regression)?

  • 1
    $\begingroup$ When you apply a nonlinear transformation to the response variable, measures of prediction accuracy--including all those you mention--are just not comparable. They mean completely different things. It is rare for more than one or two such models to be appropriate for a dataset, so perhaps what you ought to do is determine how best to express the response variable and move on from there before doing all this work. $\endgroup$
    – whuber
    Jan 14 at 21:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.