We do know that for the logistic regression, which can be modeled

$$ g( \mathbb{E} (y_i) ) = \log \left[ \frac{P(Y_i=1 |X_i = x_i)}{P(Y_i=0 |X_i = x_i)} \right] = \beta_0 + \beta_1^T x_i $$ where $g(\cdot)$ denotes the link function in the context of GLM, the $\beta_1$ and $x$ is $n$-vector, and $i$ denotes the index of each observation.

we can estimate $\hat{\beta}$using the iterative reweighted least squares method, which happens to take the form of "least squares" but basically is the Newton-Raphson method, i.e.

$$ \hat{\beta}^{(k+1)} = (X^TW^{(k)}X)^{-1}X^TW^{(k)}y^{(k)} $$

where $X$ : design matrix, $W^{(k)} = diag\{\pi_1(1-\pi_1), \cdots, \pi_n(1-\pi_n)\}$, $\pi_i = \frac{\exp(\beta_0 + \beta_1 x_i)}{1+\exp(\beta_0 + \beta_1 x_i)}$.

Since the observed information $I(\beta^{(k)})$ is

$$ I(\beta^{(k)}) =\mathbb{E} ( - l''(\hat{\beta}^{(k)})] = - l''(\hat{\beta}^{(k)}) $$

since we can analytically show that $l''(\hat{\beta}^{(k)}) = -X^TW^{(k)}X$, which is basically not the random quantity.


Now I have the model

$$ g( \mathbb{E} (y_i) ) = \beta_0 + \beta_1^T x_i $$ where $g(\mu) = \Phi^{-1}(\mu)$, where $\Phi()$ is standard normal CDF.

I'm not sure how to find the observed information in this model, and show it is indeed the random quantity, which is why do not say the Fisher scoring is the same with Newton-Raphson for probit models.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.