Why the standard errors of logistic regression are of that form? In Elements of Statistical Learning, page 125 it is written that the coefficients of a logistic regression converges to $\mathcal{N}(\beta, (X^T W X)^{-1})$ with:

*

*$X$ the $N \times (p+1)$ matrix of data

*$W$ a $N \times N$ diagonal matrix of weights with $i$th diagonal element $p(x_i;\beta)(1-p(x_i; \beta))$ where by definition $p(x_i;\beta) = \exp(\beta^T x_i)/(1+\exp(\beta^T x_i))$.

Is there a proof/intuition for the $(X^T W X)^{-1}$?
 A: The negative log likelihood for logistic regression looks like
$$ \mathcal{L}(\beta) = - \sum_{i} y_i \log\big(p(x_i;\beta)\big) + (1+y_i)\log\big(1-p(x_i;\beta)\big)$$
To get to the Hessian, we need to compute the gradient.  Note that the logistic function $p(x_i;\beta)$ satisfies the differential equation $y' = y (1-y)$ so we can use this in our expression for the derivative.  The $j^{th}$ element of the gradient is
$$ \dfrac{\partial \mathcal{L}}{\partial \beta_j} =  \sum_i y_i (1-p_i)x_{i,j} - (1-y_i)p_i x_{i,j} = \sum_{i} x_{i,j}(y_i - p_i)$$
Here I have dropped dependence of $p$ on $x$ and $\beta$ for economy of thought.  The expression presented in ESL presents the covariance matrix as an inverse of some matrix product.  Likelihood theory tells us that the sampling covariance is the inverse of the Fisher Information, so then $-X^TWX$ must be the observed information (or the hessian of the likelihood).
The second derivatives look like
$$ \dfrac{\partial^2 \mathcal{L}}{\partial\beta_k^T\partial\beta_j} = \sum_i x_{i,j} p_i(1-p_i) x_{i, k}$$
The $x$ product cross term in the summand hints towards the expression being written as a matrix product, and the weights now are clear.  I'll leave it to you to write in the proper form.
