# Understanding the GLM coefficients calculation

I was wondering how the distribution specified in a GLM changes the coefficients.

If I have understood the process, when you fit a GLM, let's say $g(E[Y|X])=X\beta$, most of the software and R packages will use a Newton Raphson algorithm to fit the $\beta$.

We have $\beta_{k+1}=\beta_k-\frac{f(\beta_k)}{f'(\beta_k)}$, where f is the gradient function. So in our case, $f(\beta)=\nabla_\beta$ (i.e. $\partial$log likelihood/$\partial\beta$) and $f'(\beta)=H$, the hessian matrix.

Again, correct me if I'm wrong, but the $H$ matrix is approximated by the Fisher information, so $H=\frac{1}{\phi}X'W_0X$, where $W_0$ is diagonal with ith elemnt equals to : $$\frac{w_i}{\phi V(\mu_i)(g'(\mu_i))^2}+\frac{V(\mu_i)g''(\mu_i)+V'(\mu_i)g'(\mu_i)}{\phi V(\mu_i)^2(g'(\mu_i))^3}$$

Here my first question, how do you estimate the variance of $\mu_i$ ?

Then,

When I want to calculate the $\nabla_\beta$, I find : $$\frac{\partial l}{\partial \beta_j}=\sum_{i=1}^n \frac{y_i-\mu_i}{\phi}*\frac{W_i}{(g^{-1})'(\eta_i)}*x_{ij}$$ Where $l$ is the log likelihood, $\phi$ the scale parameter, $\mu_i$ the estimated average, and $W$ the weight matrix.

My second question is, that all thoses elements doesn't not seem to be directly linked to the specified distribution. The only thing that looks important is the link function choosen ($g$). So I tried to launch two models on R to see if the coefficients are the same, and obviously not... for instance this code

data("UScrime")


I have two different set of coefficients.

• $\mu_i$ is a constant, so its variance is 0. However, $V(\mu_i)$ is the value of the variance function at $\mu_i$ rather than the variance of $\mu_i$. Commented Aug 11, 2016 at 12:32
• Thank you Glen_b, that was the reason why I did not understand the $V(\mu_i)$, but now it sounds so obvious I feel ridiculous... And now that I think I understand better the mechanism, this is precisly this variance function that make the difference between my two GLM's coefficients ? Commented Aug 11, 2016 at 16:19
• The variance function follows from the distribution specification. In a quasi-model you specify the variance function directly. I'll try to post an answer. Commented Aug 12, 2016 at 2:38

Note that $μ_i$ is a constant, so its variance is $0$.
However, $V(μ_i)$ is the value of the variance function at $μ_i$ rather than the variance of $μ_i$. That is, it specifies the variance component of the model: $\text{Var}(Y_i|\mathbf{x}_i)=\phi V(\mu_i)$.