0
$\begingroup$

I am performing a LOOCV linear model and I got the parameters R2 and RMSE, but I was wondering if there is a way to calculate the standard deviation of the model R2. I tried to do it in the same way I do it for k-fold cross-validation but it does not work.

This is my model

train.control <- trainControl(method = "LOOCV")
model_amb <- train(rate ~ volume, data = data_amb, method = "lm", trControl = train.control)
summary(model_amb)
sd(model_amb$resample$Rsquared)

This is what I get when I run the last line

sd(model_amb$resample$Rsquared)
[1] NA
$\endgroup$
7
  • $\begingroup$ Do you mean the standard error? The terms are not synonyms. $\endgroup$
    – Dave
    Aug 8 '21 at 3:35
  • $\begingroup$ It is the standard deviation, I usually calculate the standard deviation around the R-squared value by examining the R-squared from each fold when I do k-fold cross validation. But I think I can not do it if I am doing LOOCV, I would like to know if I can calculate that with LOOCV and if yes, how? $\endgroup$
    – Ursula
    Aug 8 '21 at 13:27
  • $\begingroup$ What happens when you try to calculate the LOOCV standard deviation the same way you calculate it for k-fold? $\endgroup$
    – Dave
    Aug 8 '21 at 13:41
  • $\begingroup$ I got NA, the code and output is above $\endgroup$
    – Ursula
    Aug 8 '21 at 13:44
  • $\begingroup$ How are you calculating model_amb\$resampleRsquared? Does that have NA values? All NA values? $\endgroup$
    – Dave
    Aug 8 '21 at 13:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.