Explain how the inverse of the information matrix and the Delta Rule be used to generate an approximate sampling variance for the estimated proportion exp(beta 0 + beta 1)/(1+exp( beta 0 + beta 1))?

For reference

I believe I am needing to find the estimated proportion of the exposed group/when x=1. I think I need to take the inverse logit with respect to beta 1 and use this in the Delta rule somehow.

Any help would be appreciated on how to do this.

  • $\begingroup$ This looks like a homework/self-study problem, in which case the self-study tag needs to be added. You also need to show your attempt(s) at solving this problem if you want a good response. $\endgroup$ – rishic Sep 29 at 6:07
  • $\begingroup$ Thanks , have edited question now. $\endgroup$ – KiwiStats Sep 29 at 6:35

[The picture does not show the complete setup, but it seems to be a logistic regression problem with $y \mid x \sim \text{Bernoulli}(\pi(x))$ where $\pi(x) = \exp[\beta_0 + \beta_1 x] / (1 + \exp[\beta_0 + \beta_1 x])$, and $x$ is a binary predictor. The model assumption is necessary to ensure that the regularity conditions of CLT for the MLE holds.]

First, we need the asymptotic distribution of the MLE $\hat \beta = (\hat \beta_0, \hat \beta_1)$. If $\beta$ is the true value of the regression parameter vector, then from the CLT of the MLE (Wikipedia) we have (the regularity conditions justifying the CLT holds under a logistic regression framework) $$ \hat \beta \stackrel{a}{\sim} N\left(\beta, I(\beta)^{-1}\right) $$

where $X \stackrel{a}{\sim} f$ means $X$ is approximately distributed as $f$, and $I(\beta)^{-1}$ is as given. (Derivation of this inverse Fisher information matrix is cumbersome. Fortunately, the final form of the matrix is given here, saving us a substantial amount of work.)

Let $g(\beta) = \exp[\beta_0 + \beta_1] / (1 + \exp[\beta_0 + \beta_1]) = 1 - 1/(1 + \exp[\beta_0 + \beta_1])$ (which is $\pi(1)$ in my notation). Then by the delta method (Wikipedia) we have ($g$ is a smooth function with infinitely many derivatives, so we don't have to worry about the regularity conditions) $$ g(\hat \beta) \stackrel{a}{\sim} N \left(g(\beta), \left( \frac{\partial g}{\partial \beta} \right) I(\beta)^{-1} \left(\frac{\partial g}{\partial \beta} \right)^T \right). $$

Here $$ \frac{\partial g}{\partial \beta} = \left( \frac{\exp[\beta_0 + \beta_1]}{(1+\exp[\beta_0 + \beta_1])^2}, \frac{\exp[\beta_0 + \beta_1]}{(1+\exp[\beta_0 + \beta_1])^2} \right) = \frac{\exp[\beta_0 + \beta_1]}{(1+\exp[\beta_0 + \beta_1])^2} (1, 1) $$ (Verify!) Use the above gradient with the given inverse Fisher information matrix $I(\beta)^{-1}$ to get the approximate variance of $g(\hat \beta)$, which is $\left( \frac{\partial g}{\partial \beta} \right) I(\beta)^{-1} \left(\frac{\partial g}{\partial \beta} \right)^T$. [You may find the identity $$ (1, 1) \begin{pmatrix} a_{11} & a_{12} \\ a_{12} & a_{22} \end{pmatrix} \begin{pmatrix} 1 \\ 1 \end{pmatrix} = a_{11} + 2 a_{12} + a_{22} $$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.