Skip to main content
replaced http://stats.stackexchange.com/ with https://stats.stackexchange.com/
Source Link

Logistic regression is not a classification algorithmnot a classification algorithm, and the decision rule you used (i.e. prob > 0.5 cutoff) is not a part of logistic regression model.

Logistic regression predicts conditional probabilities of successes, so you should instead calculate your errors accordingly, i.e. comparing to the probabilities, not to the probabilities rounded to {0, 1}.

Below you can see an example where errors from leave-one-out cross-validation are calculated for model using your decision rule, and the ordinary errors for logistic regression. As you can see, cv.glm uses the second approach.

fit <- glm(vs ~ mpg, data = mtcars, family = binomial)
out <- NULL
for (i in 1:nrow(mtcars))
  out[i] <- predict(update(fit, data = mtcars[-i,]), newdata = mtcars[i,], type = "response")

boot::cv.glm(mtcars, fit)$delta[1]
## 0.1497903
mean((mtcars$vs - round(out))^2)
## 0.1875
mean((mtcars$vs - out)^2)
## 0.1497903

Logistic regression is not a classification algorithm, and the decision rule you used (i.e. prob > 0.5 cutoff) is not a part of logistic regression model.

Logistic regression predicts conditional probabilities of successes, so you should instead calculate your errors accordingly, i.e. comparing to the probabilities, not to the probabilities rounded to {0, 1}.

Below you can see an example where errors from leave-one-out cross-validation are calculated for model using your decision rule, and the ordinary errors for logistic regression. As you can see, cv.glm uses the second approach.

fit <- glm(vs ~ mpg, data = mtcars, family = binomial)
out <- NULL
for (i in 1:nrow(mtcars))
  out[i] <- predict(update(fit, data = mtcars[-i,]), newdata = mtcars[i,], type = "response")

boot::cv.glm(mtcars, fit)$delta[1]
## 0.1497903
mean((mtcars$vs - round(out))^2)
## 0.1875
mean((mtcars$vs - out)^2)
## 0.1497903

Logistic regression is not a classification algorithm, and the decision rule you used (i.e. prob > 0.5 cutoff) is not a part of logistic regression model.

Logistic regression predicts conditional probabilities of successes, so you should instead calculate your errors accordingly, i.e. comparing to the probabilities, not to the probabilities rounded to {0, 1}.

Below you can see an example where errors from leave-one-out cross-validation are calculated for model using your decision rule, and the ordinary errors for logistic regression. As you can see, cv.glm uses the second approach.

fit <- glm(vs ~ mpg, data = mtcars, family = binomial)
out <- NULL
for (i in 1:nrow(mtcars))
  out[i] <- predict(update(fit, data = mtcars[-i,]), newdata = mtcars[i,], type = "response")

boot::cv.glm(mtcars, fit)$delta[1]
## 0.1497903
mean((mtcars$vs - round(out))^2)
## 0.1875
mean((mtcars$vs - out)^2)
## 0.1497903
Source Link
Tim
  • 141.2k
  • 26
  • 270
  • 512

Logistic regression is not a classification algorithm, and the decision rule you used (i.e. prob > 0.5 cutoff) is not a part of logistic regression model.

Logistic regression predicts conditional probabilities of successes, so you should instead calculate your errors accordingly, i.e. comparing to the probabilities, not to the probabilities rounded to {0, 1}.

Below you can see an example where errors from leave-one-out cross-validation are calculated for model using your decision rule, and the ordinary errors for logistic regression. As you can see, cv.glm uses the second approach.

fit <- glm(vs ~ mpg, data = mtcars, family = binomial)
out <- NULL
for (i in 1:nrow(mtcars))
  out[i] <- predict(update(fit, data = mtcars[-i,]), newdata = mtcars[i,], type = "response")

boot::cv.glm(mtcars, fit)$delta[1]
## 0.1497903
mean((mtcars$vs - round(out))^2)
## 0.1875
mean((mtcars$vs - out)^2)
## 0.1497903