How to estimate coefficients of logistic model

Question

Consider model $logit(p)=a+bx$. I would like to get an analytic formula of $a$ and $b$ like in linear regression. In linear regression, we can get a formula of estimates of $a$ and $b$.

I tried using MLE. But it is too complicated for me.

I am doing research about the statistical genetics. Here $p$ is penetrance and $x$ is genotype. In linear regression, we can use the coefficient $b$ to represent the covariance matrix of $x$ and trait $y$ (both of them should be standardized). I am thinking whether we can use the $b$ in logistic model to recover the information about the covariance matrix of $x$ and $y$ (binary trait) in meta-analysis (they do not provide individual level data). But some cohort will prove the odds ratio and coefficient t-statistic. I do not know how to use it to recover the covariance matrix of $x$ and $y$.

But some summary statistics will be provided like estimated odds ratio, Coefficient t-statistic.

Answer 1

I strongly encourage you to work through the case where $x$ can take two values (e.g. 0 = dominant, 1 = recessive as in your example), as it a useful exercise for understanding the logistic model and also the MLE approach.

The model in question is supposed to be $$ Y_i \sim \mathrm{Bernoulli}(p_i), $$ where $p_i = \mathrm{logistic}(a + bx_i)$, $i = 1,\ldots,n$. Suppose the observed values are $y_1, \ldots, y_n$.

Homework:

1. What is the probability of observing this particular string of $y_i$ (assuming they are all independent). Hint: split according to whether $y_i = 0$ or $1$ and similarly for $x_i$.
2. What is the log of this?
3. What happens to the log likelihood if $a$ gets large or small (negative)? Same question for $b$.
4. Conclude that you can maximise the log likelihood by differentiating in $a$ and $b$ then finding where the derivative is zero (separately for $a$ and $b$).

OK, for the final answer: Write $\#_{l,m} = \#\{i : y_i = l, x_i = m\}$, so $\#_{0,1}$ is the number of observations with $y_i = 0$ and $x_i = 1$ etc.

You'll see the problem can be parameterised more easily in terms of $$ p = \frac{e^{a}}{1 + e^{a}} $$ and $$ r = \frac{e^{a+b}}{1 + e^{a+b}}. $$

Assuming I didn't make a mistake, we find $$ r = \frac{\#_{1,1}}{\#_{0,1} + \#_{1,1}}, $$ i.e. the fraction of cases with $y_i = 1$ when the independent variable has $x_i = 1$.

Then, $$ p = \frac{\#_{1,0}}{\#_{0,0} + \#_{1,0}} $$ is the fraction when $x_i = 0$.

So, $a + b = \ln(r/(1-r))$ is the log odds for $y_i = 1$ when $x_i = 1$.

Finally, $a = \ln(p/(1-p))$ and $b = \ln(r(1-p)/p(1-r))$ is the log odds ratio.