From the training output below, it looks like R squared is not calculated by the traditional formula 1 - SSE/SST, since lower error rate has lower R squared. So how is it calculated?

  colsample_bytree  min_child_weight  RMSE      Rsquared 
  0.4               3                 16963.39  0.8799191
  0.4               5                 16813.24  0.8788395

2 Answers 2


The code is here:

> R2
function(pred, obs, formula = "corr", na.rm = FALSE) {
    n <- sum(complete.cases(pred))
           corr = cor(obs, pred, use = ifelse(na.rm, "complete.obs", "everything"))^2,
           traditional = 1 - (sum((obs-pred)^2, na.rm = na.rm)/((n-1)*var(obs, na.rm = na.rm))))

It follows the idea of calculating R then squaring it. There are a lot of formulas for R^2 that can be used. See Kvalseth. Cautionary note about R^2. American Statistician (1985) vol. 39 (4) pp. 279-285. All of this is described at ?R2.


Update: The R2 function is deprecated. You can get R2 by the postResample function now.

The code is here and shows in line 136 and 142 the formula:

resamplCor <- try(cor(pred, obs, use = "pairwise.complete.obs"), silent = TRUE
out <- c(sqrt(mse), resamplCor^2, mae)

So postResample uses the squared correlation to calculate R2.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.