0
$\begingroup$

I was wondering how the distribution specified in a GLM changes the coefficients.

If I have understood the process, when you fit a GLM, let's say $g(E[Y|X])=X\beta$, most of the software and R packages will use a Newton Raphson algorithm to fit the $\beta$.

We have $\beta_{k+1}=\beta_k-\frac{f(\beta_k)}{f'(\beta_k)}$, where f is the gradient function. So in our case, $f(\beta)=\nabla_\beta$ (i.e. $\partial$log likelihood/$\partial\beta$) and $f'(\beta)=H$, the hessian matrix.

Again, correct me if I'm wrong, but the $H$ matrix is approximated by the Fisher information, so $H=\frac{1}{\phi}X'W_0X$, where $W_0$ is diagonal with ith elemnt equals to : $$\frac{w_i}{\phi V(\mu_i)(g'(\mu_i))^2}+\frac{V(\mu_i)g''(\mu_i)+V'(\mu_i)g'(\mu_i)}{\phi V(\mu_i)^2(g'(\mu_i))^3}$$

Here my first question, how do you estimate the variance of $\mu_i$ ?

Then,

When I want to calculate the $\nabla_\beta$, I find : $$ \frac{\partial l}{\partial \beta_j}=\sum_{i=1}^n \frac{y_i-\mu_i}{\phi}*\frac{W_i}{(g^{-1})'(\eta_i)}*x_{ij} $$ Where $l$ is the log likelihood, $\phi$ the scale parameter, $\mu_i$ the estimated average, and $W$ the weight matrix.

My second question is, that all thoses elements doesn't not seem to be directly linked to the specified distribution. The only thing that looks important is the link function choosen ($g$). So I tried to launch two models on R to see if the coefficients are the same, and obviously not... for instance this code

data("UScrime")
model1=glm(formula=Prob~M+So+Ed,data = UScrime,family=gaussian(link=log))
model2=glm(formula=Prob~M+So+Ed,data = UScrime,family=Gamma(link=log))

I have two different set of coefficients.

Thanks for you answers

$\endgroup$
  • $\begingroup$ $\mu_i$ is a constant, so its variance is 0. However, $V(\mu_i)$ is the value of the variance function at $\mu_i$ rather than the variance of $\mu_i$. $\endgroup$ – Glen_b Aug 11 '16 at 12:32
  • $\begingroup$ Thank you Glen_b, that was the reason why I did not understand the $V(\mu_i)$, but now it sounds so obvious I feel ridiculous... And now that I think I understand better the mechanism, this is precisly this variance function that make the difference between my two GLM's coefficients ? $\endgroup$ – Arault Aug 11 '16 at 16:19
  • $\begingroup$ The variance function follows from the distribution specification. In a quasi-model you specify the variance function directly. I'll try to post an answer. $\endgroup$ – Glen_b Aug 12 '16 at 2:38
1
$\begingroup$

The notation may be a little misleading.

Note that $μ_i$ is a constant, so its variance is $0$.

However, $V(μ_i)$ is the value of the variance function at $μ_i$ rather than the variance of $μ_i$. That is, it specifies the variance component of the model: $\text{Var}(Y_i|\mathbf{x}_i)=\phi V(\mu_i)$.

The variance function follows from the distribution specification in a GLM.

In a quasi-model you'd specify the variance function directly.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.