# Leave-one-out cross validation output interpretation and ROC curve

I have taken plenty of time to try and help myself, but I keep reaching dead ends.

I have a dataset consisting of body measurements collected from a bird species, and the sex of each bird (known by molecular means). I built a logistic regression model (using the AIC information criterion) to assess which measurements explain better the sex of the birds. My ultimate goal is to have an equation which could be used by others under field conditions to predict reliably the sex of the birds by taking as few body measurements as possible.

My final model includes four independent variables, namely "Culmen", "Head-bill", "Tarsus length", and "Wing length" (all continuous). I wish my model was a little more parsimonious, but all the variables seem to be important according to AIC criterion. Because the model produced should be used as prediction tool, I decided validate it using a leave-one-out cross validation approach. In my learning process, I first tried to complete the analyses (cross-validation and plotting) by including only one explanatory variable, namely "Culmen".

The output of the cross validation (package "boot" in R) yields two values (deltas), which are the cross-validated prediction errors where the first number is the raw leave-one-out, or lieu cross-validation result, and the second one is a bias-corrected version of it.

model.full <- glm(Sex ~ Culmen, data = my.data, family = binomial)
summary(model.full.1)

cv.glm(my.data, model.full, K=114)

$call cv.glm(data = my.data, glmfit = model.full, K = 114)$K
[1] 114

$delta [1] 0.05941851 0.05937288  Q1. Could anyone expalin what do these two values represent and how to interpret them? Following is the code as presented by Dr. Markus Müller (Calimo) in a similar, albeit not identical, post (https://stackoverflow.com/questions/20346568/feature-selection-cross-validation-but-how-to-make-roc-curves-in-r) which I tried to tweak to meet my data: library(pROC) data(my.data) k <- 114 # Number of observations or rows in dataset n <- dim(my.data)[1] indices <- sample(rep(1:k, ceiling(n/k))[1:n]) all.response <- all.predictor <- aucs <- c() for (i in 1:k) { test = my.data[indices==i,] learn = my.data[indices!=i,] model <- glm(Sex ~ Culmen, data = learn, family=binomial) model.pred <- predict(model, newdata=test) aucs <- c(aucs, roc(test$Sex, model.pred)$auc) all.response <- c(all.response, test$outcome)
all.predictor <- c(all.predictor, model.pred)
}