# Chi Square vs F Tests for GLM Model Comparisons

I've been creating some models in R using glm() and rxGlm(). I'm experienced in building GLMs but my memory of some of the underlying theory is a little rusty.

I'm interested in comparing model fits for nested models using chi-square tests, F tests, etc.

I'm able to compare nested glm model objects using

anova (model1, model2, test = "Chisq")


etc. From reading around the subject a little, it seems that chi-square is only valid for certain GLMs - those where the scale parameter is fixed (Poisson & binomial), whereas the F test should be used where the scale parameter is estimated (eg normal, gamma). Is this correct?

I have a particular interest in creating GLMs using the Tweedie family of distributions. Is this a case where F would be preferable to chi-square?

## 1 Answer

Basically, yes. $$F$$ is used when the dispersion parameter is estimated rather than assumed to be fixed to some known value. $$F$$ is also often used for quasi-likelihood models where a somewhat ad hoc overdispersion parameter is estimated (see e.g. Venables and Ripley Modern Applied Statistics with S).

For a Tweedie distribution you're estimating a dispersion parameter and (possibly) a shape parameter. If you estimate the shape parameter rather than fixing it some a priori value, you'll probably be underestimating the uncertainty unless you do something fancy (e.g. bootstrapping).

• Thank you. For the Tweedie application I'm using a fixed a priori shape parameter for the time being although I know that techniques exist for estimating it. I've done some anova() calls using test = "F" instead of test = "Chisq" and I'm getting identical p-values (for a number of different GLM error structures - Poisson, gaussian, gamma, tweedie). I didn't expect this - should I have? – Alan May 28 '19 at 11:38
• Also for fitting GLMs I often have to use the rxGlm() function in the revoScaleR package because it performs so much better with large data sets. However the objects that the rxGlm() function returns don't seem to work with the anova() function, so that: anova(rxModel1, rxModel2, test = "whatever") and anova(as.glm(rxModel1), as.glm(rxModel2), test = "whatever") both fail. Is there a way around this, I wonder? Apologies - I haven't managed to format this comment properly (not sure how!) – Alan May 28 '19 at 11:39
• Huh. For your first question, I'm surprised too. Can you post a reproducible example? For your second question: there's an unanswered question about this on SO ... (oh, never mind, I see that it's you ...) I don't have Microsoft R installed and don't really plan to, so it's going to be hard for me to find a solution for you ...) – Ben Bolker May 28 '19 at 13:26