Skip to main content
2 of 2
replaced http://stats.stackexchange.com/ with https://stats.stackexchange.com/

Logistic regression is not a classification algorithm, and the decision rule you used (i.e. prob > 0.5 cutoff) is not a part of logistic regression model.

Logistic regression predicts conditional probabilities of successes, so you should instead calculate your errors accordingly, i.e. comparing to the probabilities, not to the probabilities rounded to {0, 1}.

Below you can see an example where errors from leave-one-out cross-validation are calculated for model using your decision rule, and the ordinary errors for logistic regression. As you can see, cv.glm uses the second approach.

fit <- glm(vs ~ mpg, data = mtcars, family = binomial)
out <- NULL
for (i in 1:nrow(mtcars))
  out[i] <- predict(update(fit, data = mtcars[-i,]), newdata = mtcars[i,], type = "response")

boot::cv.glm(mtcars, fit)$delta[1]
## 0.1497903
mean((mtcars$vs - round(out))^2)
## 0.1875
mean((mtcars$vs - out)^2)
## 0.1497903
Tim
  • 141.2k
  • 26
  • 270
  • 512