-2
$\begingroup$

I am doing a study about logistic regression. I have to write a program for the admission process of the school. The result is passed or failed.

Logit = L = b0 + b1X1 + b2X2 + …+ bkXk ...

What is the formula for b0, b1, b2, b3 (i.e., the beta coefficients) that can be seen at the picture? What is the formula for the raw coefficient for logistic regression?

finding the formula that outputs these variables

$\endgroup$
  • 3
    $\begingroup$ The coefficients are the solutions of the maximum likelihood equations (see 2.1.2 of the link below), there is no closed form formula for them, they are computed via iterative procedures (e.g. see 2.1.3 of the link) czep.net/stat/mlelr.pdf $\endgroup$ – user83346 Feb 21 '16 at 8:09
  • $\begingroup$ see stats.stackexchange.com/a/30884/70282 $\endgroup$ – Chris Oct 31 '16 at 17:26
  • $\begingroup$ A question asking about how the betas are calculated is not really a question asking for code. This should be considered on topic here, IMO. $\endgroup$ – gung Nov 2 '16 at 21:30
  • $\begingroup$ The question at stats.stackexchange.com/questions/94 asks for characterizations of the (very special) cases where a formula might exist. $\endgroup$ – whuber Nov 3 '16 at 17:00
1
$\begingroup$

See https://stats.stackexchange.com/a/30884/70282, you'll notice you can divide or multiple by the std dev of the predictor variable to go back and forth.

Example:

d=data.frame(x1=runif(1000,10,20),
             x2=runif(1000,100,200),
             x3=runif(1000,0,100))
d$y=ifelse(d$x1>15 & d$x2>150 & d$x3>50,1,0)
summary(d)
d2=as.data.frame(scale(d))
d2$y=d$y
summary(m <- glm(y~.,d,family = 'binomial'))
summary(m2 <- glm(y~.,d2,family = 'binomial'))
coef(m) #metric coeff
coef(m2) #standarized coef
coef(m)*c(1,sd(d$x1),sd(d$x2),sd(d$x3)) #standardized from metric
coef(m2)/c(1,sd(d$x1),sd(d$x2),sd(d$x3)) #metric from standardized

As a function:

logistic.beta=function(m){
  coef(m)[-1]*sapply(m$data,sd)[names(coef(m))[-1]]
}
$\endgroup$
1
$\begingroup$

As mentioned by @fcop, there is no formula (analytical solution) for the $\beta$. Here is an example on how the coefficients are calculated using iterative methods. (Comparing to R glm function)

x=as.matrix(mtcars[,c("wt","hp")]) y=mtcars$am

logistic_loss <- function(w){
  p=plogis(x %*% w)
  L=-y*log(p)-(1-y)*log(1-p)
  sum(L)
}

logistic_loss_gr <- function(w){
  p=plogis(x %*% w)
  v=t(x) %*% (p - y)
  c(v)
}

res=optim(c(0.1,0.1),fn=logistic_loss, gr = logistic_loss_gr ,method="BFGS")

res$p

glm(am~wt+hp-1,mtcars,family=binomial())$coefficients

enter image description here

The coeffcients are pretty close but not exact the same, see this post for explaination.

R using GLM and manual solve logistic regression have different (close but not exactly the same) results

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.