0
$\begingroup$

An answer to a previous question on how to manually calculate the MLE for beta coefficients from a logistic regression was given the below answer (see previous question here: How to manually calculate the intercept and coefficient in logistic regression).

Take for example, the crosstab bellow shows how many males/females are in the honor class.

           |         female
       hon |      male     female |     Total  
-----------+----------------------+----------
         0 |        74         77 |       151 
         1 |        17         32 |        49 
-----------+----------------------+----------
     Total |        91        109 |       200

As mentioned above $∑_iy_ix_{ij}=∑_ip_ix_{ij}$ holds. The left hand side(LHS) is the expectation of the observations (y's in samples) and the right hand side(RHS) is the model's expectation.

Assuming the function is $log(\frac{p1}{p1−p})=β_0+β_1x_i$ or equivalently $p=(\frac{1}{1+e^{β0+β1*x_i}})$ ($x_i$ represents the feature of the observation being a female, it is 1 if the observation is a female and 0 otherwise), obviously we know that the following two equations hold respectively when $X=1$ and when $X=0$ with the data shown above:

$\frac{32}{109}=(\frac{1}{1+e^{β0+β1*x_1}})$

$\frac{17}{91}=(\frac{1}{1+e^{β0+β1*x_0}})$

So the intercept $(β_0)$ is -1.47 and the coefficient $(β_1)$ is 0.593. You can manually get it.

I am confused as to how they jumped from $\frac{32}{109}$ to getting the estimate for $(β_1)$. This seems like a useful way to estimate MLEs of parameters in a logistic regression. Can anyone help explain how they got to $(β_1)$ is 0.593 ?

$\endgroup$
3
$\begingroup$

In this regression, then are only two observations and two fitted values, which are $\hat p_m=17/91$ for males and $\hat p_f=32/109$ for females.

Converting to the logit linear predictor scale gives: $$\mbox{logit}(\hat p_m)= \hat\beta_0=\log\frac{ 17}{91-17} = \log\frac{17}{74}=-1.471$$ and $$\mbox{logit}(\hat p_f)=\hat\beta_0+\hat\beta_1=\log\frac{32}{109-32} = \log\frac{32}{77}=-0.878$$ Hence $$\hat\beta_1= -0.878 + 1.471 = 0.593$$

Note that this type of closed-form calculation only works for logistic regressions with a single categorical factor, or for models that are equivalent to regressions with a single categorical factor, such as factorial models with all possible interactions. Logistic regressions with additive non-interacting terms do not admit closed-form estimators.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.