I have a question regarding the validity of using RMSE (Root Mean Squared Error) to compare different logistic models. The response is either 0
or 1
and the predictions are probabilities between 0
-1
?
Is the way applied below valid with the binary responses also?
##### Using glmnet
require(glmnet)
load(url("https://github.com/cran/glmnet/raw/master/data/BinomialExample.RData"))
cvfit = cv.glmnet(x, y, family="binomial", type.measure="mse")
A = predict(cvfit, newx=x, s="lambda.min", type="response")
RMSE1 = mean((y - A)^2)
# 0.05816881
##### glm
mydata = read.csv("https://stats.idre.ucla.edu/stat/data/binary.csv")
mydata$rank = factor(mydata$rank)
mylogit = glm(admit~gre+gpa+rank, data=mydata, family="binomial")
AAA = predict(mylogit, newdata=mydata, type="response")
RMSE2 = mean((mydata$admit - AAA)^2)
# 0.194714