I have a question regarding the validity of using RMSE (Root Mean Squared Error) to compare different logistic models. The response is either 0
or 1
and the predictions are probabilities between 0
-1
?
Is the way applied below valid with the binary responses also?
# Using glmnet
require(glmnet)
load(url("https://github.com/cran/glmnet/raw/master /data/BinomialExample.RData"))
cvfit = cv.glmnet(x, y, family = "binomial", type.measure = "mse")
A <- predict(cvfit, newx = x, s = "lambda.min", type = "response")
RMSE1 <- mean((y - A)^2)
# 0.05816881
# glm
mydata <- read.csv("http://www.ats.ucla.edu/stat/data/binary.csv")
mydata$rank <- factor(mydata$rank)
mylogit <- glm(admit ~ gre + gpa + rank, data = mydata, family = "binomial")
AAA <- predict(mylogit, newdata = mydata, type = "response")
RMSE2 <- mean((mydata$admit - AAA)^2)
# 0.194714