Consider model $logit(p)=a+bx$. I would like to get an analytic formula of $a$ and $b$ like in linear regression. In linear regression, we can get a formula of estimates of $a$ and $b$.

I tried using MLE. But it is too complicated for me.

I am doing research about the statistical genetics. Here $p$ is penetrance and $x$ is genotype. In linear regression, we can use the coefficient $b$ to represent the covariance matrix of $x$ and trait $y$ (both of them should be standardized). I am thinking whether we can use the $b$ in logistic model to recover the information about the covariance matrix of $x$ and $y$ (binary trait) in meta-analysis (they do not provide individual level data). But some cohort will prove the odds ratio and coefficient t-statistic. I do not know how to use it to recover the covariance matrix of $x$ and $y$.

But some summary statistics will be provided like estimated odds ratio, Coefficient t-statistic.

  • $\begingroup$ Are you saying $x$ is a binary variable? (This is basically the only situation where there is an explicit solution) $\endgroup$ Oct 25 '15 at 17:15
  • $\begingroup$ @P.Windridge No. $x$ is the genotype. It can be $0,1,2$ (additive model)or $0,1$ (dominance or recessive model ) depending what model we use. I am pleased if you can put you solution here. $\endgroup$
    – Mike Brown
    Oct 25 '15 at 17:18

I strongly encourage you to work through the case where $x$ can take two values (e.g. 0 = dominant, 1 = recessive as in your example), as it a useful exercise for understanding the logistic model and also the MLE approach.

The model in question is supposed to be $$ Y_i \sim \mathrm{Bernoulli}(p_i), $$ where $p_i = \mathrm{logistic}(a + bx_i)$, $i = 1,\ldots,n$. Suppose the observed values are $y_1, \ldots, y_n$.


  1. What is the probability of observing this particular string of $y_i$ (assuming they are all independent). Hint: split according to whether $y_i = 0$ or $1$ and similarly for $x_i$.
  2. What is the log of this?
  3. What happens to the log likelihood if $a$ gets large or small (negative)? Same question for $b$.
  4. Conclude that you can maximise the log likelihood by differentiating in $a$ and $b$ then finding where the derivative is zero (separately for $a$ and $b$).

OK, for the final answer: Write $\#_{l,m} = \#\{i : y_i = l, x_i = m\}$, so $\#_{0,1}$ is the number of observations with $y_i = 0$ and $x_i = 1$ etc.

You'll see the problem can be parameterised more easily in terms of $$ p = \frac{e^{a}}{1 + e^{a}} $$ and $$ r = \frac{e^{a+b}}{1 + e^{a+b}}. $$

Assuming I didn't make a mistake, we find $$ r = \frac{\#_{1,1}}{\#_{0,1} + \#_{1,1}}, $$ i.e. the fraction of cases with $y_i = 1$ when the independent variable has $x_i = 1$.

Then, $$ p = \frac{\#_{1,0}}{\#_{0,0} + \#_{1,0}} $$ is the fraction when $x_i = 0$.

So, $a + b = \ln(r/(1-r))$ is the log odds for $y_i = 1$ when $x_i = 1$.

Finally, $a = \ln(p/(1-p))$ and $b = \ln(r(1-p)/p(1-r))$ is the log odds ratio.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.