I am doing a study about logistic regression. I have to write a program for the admission process of the school. The result is passed or failed.
Logit = L = b0 + b1X1 + b2X2 + …+ bkXk ...
What is the formula for b0, b1, b2, b3 (i.e., the beta coefficients) that can be seen at the picture? What is the formula for the raw coefficient for logistic regression?
See https://stats.stackexchange.com/a/30884/70282, you'll notice you can divide or multiple by the std dev of the predictor variable to go back and forth.
Example:
d=data.frame(x1=runif(1000,10,20),
x2=runif(1000,100,200),
x3=runif(1000,0,100))
d$y=ifelse(d$x1>15 & d$x2>150 & d$x3>50,1,0)
summary(d)
d2=as.data.frame(scale(d))
d2$y=d$y
summary(m <- glm(y~.,d,family = 'binomial'))
summary(m2 <- glm(y~.,d2,family = 'binomial'))
coef(m) #metric coeff
coef(m2) #standarized coef
coef(m)*c(1,sd(d$x1),sd(d$x2),sd(d$x3)) #standardized from metric
coef(m2)/c(1,sd(d$x1),sd(d$x2),sd(d$x3)) #metric from standardized
As a function:
logistic.beta=function(m){
coef(m)[-1]*sapply(m$data,sd)[names(coef(m))[-1]]
}
As mentioned by @fcop, there is no formula (analytical solution) for the $\beta$. Here is an example on how the coefficients are calculated using iterative methods. (Comparing to R glm function)
x=as.matrix(mtcars[,c("wt","hp")]) y=mtcars$am
logistic_loss <- function(w){
p=plogis(x %*% w)
L=-y*log(p)-(1-y)*log(1-p)
sum(L)
}
logistic_loss_gr <- function(w){
p=plogis(x %*% w)
v=t(x) %*% (p - y)
c(v)
}
res=optim(c(0.1,0.1),fn=logistic_loss, gr = logistic_loss_gr ,method="BFGS")
res$p
glm(am~wt+hp-1,mtcars,family=binomial())$coefficients
The coeffcients are pretty close but not exact the same, see this post for explaination.
