GLM coefficient estimates distribution

Question

I'm confused about how the GLM coefficients are distributed. Let $\hat\beta$ be the vector of coefficient estimates and $n$ the number of observations. Are the coefficients distributed like this:

$$ \hat\beta \sim N\bigg(\beta, \frac{1}{n} \bigg(E\Big[-\frac{\partial^2 \mathcal{L}(\beta)}{\partial \beta \partial \beta^T}\Big]\bigg)^{-1}\bigg), $$

or are they distributed like that:

$$ \hat\beta \sim N\bigg(\beta, \bigg(E\Big[-\frac{\partial^2 \mathcal{L}(\beta)}{\partial \beta \partial \beta^T}\Big]\bigg)^{-1}\bigg)? $$

Any insight would be very helpful, as I'm having problems with understanding this.

Answer 1

Assuming $\hat{\beta}$ is the MLE, we do not know the distribution of $\hat{\beta}$ but we do know the asymptotic distribution of $\sqrt n (\hat{\beta} - \beta)$ to be $N\bigg(0, \bigg(E\Big[-\frac{\partial^2 \mathcal{L}(\beta)}{\partial \beta \partial \beta^T}\Big]\bigg)^{-1}\bigg)$.

Rearranging terms we say $\hat\beta$ is approximately $ N\bigg(\beta, \frac{1}{n} \bigg(E\Big[-\frac{\partial^2 \mathcal{L}(\beta)}{\partial \beta \partial \beta^T}\Big]\bigg)^{-1}\bigg)$. We cannot say this is an asymptotic distribution because as $ n \rightarrow \infty$ the variance will be zero.

Answer 2

Your question is, imho, not directly related to GLMs but can be answered using more simple examples that do not require likelihood approaches:

Take $X_i\sim iid (\mu,1)$, i.e., the $X_i$ are independently and identically distributed from a distribution with some mean $\mu$ and a variance of 1 (for simplicity).

Then, by the central limit theorem we know that as $n\to\infty$, $$\sqrt{n}(\bar{X}-\mu)\to_dN(0,1)$$

Basically, we "magnify" the difference $\bar{X}-\mu$ which itself, by the law of large numbers, vanishes as $n\to\infty$.

We can then approximately say that $\bar{X}\sim N(\mu,1/n)$. The "logic" is as follows:

$$Var(\sqrt{n}(\bar{X}-\mu))=1\Rightarrow nVar(\bar{X}-\mu)=1\Rightarrow Var(\bar{X}-\mu)=1/n\Rightarrow Var(\bar{X})=1/n,$$ where the last implication is because $\mu$ is nonrandom. This is only an approximation because the leftmost statement is the asymptotic variance, i.e., the variance of $\sqrt{n}(\bar{X}-\mu)$ as $n\to\infty$, so that the rightmost statement ought to consequently read $Var(\bar{X})=0$. That, by the LLN, is correct, but the approximation often turns out to be useful nevertheless.