26
$\begingroup$

My question relates mostly around the practical differences between General Linear Modeling (GLM) and Generalized Linear Modelling (GZLM).

In my case it would be a few continuous variables as covariates and a few factors in an ANCOVA, versus GZLM. I want to examine the main effects of each variable, as well as one three-way interaction that I will outline in the model. I can see this hypothesis being tested in an ANCOVA, or using GZLM. To some extent I understand the math processes and reasoning behind running a General Linear Model like an ANCOVA, and I somewhat understand that GZLMs allow for a link function connecting the linear model and the dependent variable (ok, I lied, maybe I don't really understand the math).

What I really don't understand are the practical differences or reasons for running one analysis and not the other when the probability distribution used in the GZLM is normal (i.e., identity link function?). I get very different results when I run one over the other. Could I run either? My data is somewhat non-normal, but works to some extent both in the ANCOVA and the GZLM. In both cases my hypothesis is supported, but in the GZLM the p value is "better".

My thought was that an ANCOVA is a linear model with a normally distributed dependent variable using an identity link function, which is exactly what I can input in a GZLM, but these are still different.

Please shed some light on these questions for me, if you can!

Based on the first answer I have the additional question:

If they are identical except for the significance test that it utilized (i.e., F test vs. Wald Chi Square), which would be most appropriate to use? ANCOVA is the "go-to method", but I am unsure why the F test would be preferable. Can someone shed some light on this question for me?

Improve this question
$\endgroup$
1
  • $\begingroup$ @onestop's answer is good; I upvoted it long ago. To get a clearer sense of the connection between the general linear model & the generalized linear model, it may help you to read my answer here: difference-between-logit-and-probit-models (although it was written in a different context). Assuming your errors are normally distributed, but the error variance is not known a-priori, the $t$ & $F$ tests that software will return w/ an ANCOVA will be correct; the p-value from the Wald test will be too low, unless your N is very large. $\endgroup$
    – gung - Reinstate Monica
    Commented Jul 29, 2013 at 3:02

2 Answers 2

Reset to default
24
$\begingroup$

A generalized linear model specifying an identity link function and a normal family distribution is exactly equivalent to a (general) linear model. If you're getting noticeably different results from each, you're doing something wrong.

Note that specifying an identity link is not the same thing as specifying a normal distribution. The distribution and the link function are two different components of the generalized linear model, and each can be chosen independently of the other (although certain links work better with certain distributions, so most software packages specify the choice of links allowed for each distribution).

Some software packages may report noticeably different $p$-values when the residual degrees of freedom are small if it calculates these using the asymptotic normal and chi-square distributions for all generalized linear models. All software will report $p$-values based on Student's $t$- and Fisher's $F$-distributions for general linear models, as these are more accurate for small residual degrees of freedom as they do not rely on asymptotics. Student's $t$- and Fisher's $F$-distributions are strictly valid for the normal family only, although some other software for generalized linear models may also use these as approximations when fitting other families with a scale parameter that is estimated from the data.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you for your reply! I am glad to hear your answer, as that was my original assumption. I was told otherwise by a professor at my institution, so I really had to go digging. I am using SPSS and I now see that the parameter estimates are indeed identical (e.g., B values). I see now that what had confused me originally were the different p values. The ANCOVA statistic is based on a standard F test, while the GZLM is based on the Wald Chi-Square, correct? I have read that the Wald Chi-Square is used when you use parameters of the sample (such as in the GZLM). $\endgroup$
    – Behacad
    Commented Feb 15, 2011 at 21:31
5
$\begingroup$

I would like to include my experience in this discussion. I have seen that a generalized linear model (specifying an identity link function and a normal family distribution) is identical to a general linear model only when you use the maximum likelihood estimate as scale parameter method. Otherwise if "fixed value = 1" is chosen as scale parameter method you get very different p values. My experience suggest that usually "fixed value = 1" should be avoided. I'm curious to know if someone knows when it is appropriate to choose fixed value = 1 as scale parameter method. Thanks in advance. Mark

Improve this answer
$\endgroup$
1
  • 3
    $\begingroup$ Typically one uses a fixed scale only with models like logistic regression or Poisson regression, where the response is a count or indicator/frequency variable. In this case there is no analogue to the scale parameter in normal regression. $\endgroup$
    – Hong Ooi
    Commented Feb 16, 2011 at 4:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.