# What are the differences between ANOVAs and GLMs?

I'm wondering how different are generalized linear model and ANOVA. I thought it was because of categorical vs continuous variables, but it doesn't seems to be the case.

Also, I know that it's possible to add mixed effects in a GLMM. Is it the case for ANOVAs?

So, why would one use GLM vs ANOVAs?

Here is an example:

# Difference regression vs ANOVA vs GLM
gr1 = rnorm(10,10,1)
gr2 = rnorm(10,20,1)

df=data.frame(nm = c(gr1,gr2),
gr = c(rep(0,length(gr1)),
rep(1,length(gr2)))) # ANOVA

summary(aov(nm~gr, data = df))

Df Sum Sq Mean Sq F value  Pr(>F)
gr           1  496.4   496.4   621.1 2.1e-15 ***
Residuals   18   14.4     0.8
---
Signif. codes:
0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1


# GLM

glm(nm~gr,family = "gaussian", data = df)

Call:  glm(formula = nm ~ gr, family = "gaussian", data = df)

Coefficients:
(Intercept)           gr
10.050        9.964

Degrees of Freedom: 19 Total (i.e. Null);  18 Residual
Null Deviance:      510.8
Residual Deviance: 14.39    AIC: 56.17


And once we are there, why not comparing linear regressions and t-test!

# Linear regression

summary(lm(nm~gr, data = df))

Residuals:
Min       1Q   Median       3Q      Max
-1.13990 -0.85335  0.00061  0.41865  1.92899

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept)  10.0502     0.2827   35.55  < 2e-16 ***
gr            9.9638     0.3998   24.92  2.1e-15 ***
---
Signif. codes:
0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.894 on 18 degrees of freedom
Multiple R-squared:  0.9718,    Adjusted R-squared:  0.9703
F-statistic: 621.1 on 1 and 18 DF,  p-value: 2.095e-15


# t-test

tt=t.test(df$$nm~df$$gr);tt
Welch Two Sample t-test

data:  df$$nm by df$$gr
t = -24.922, df = 17.914, p-value = 2.347e-15
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
-10.804012  -9.123526
sample estimates:
mean in group 0 mean in group 1
10.05023        20.01400

(tt$statistic)^2 t 621.0907  • ANOVA assumes Gaussian distribution of the residuals (and uses a linear model that minimizes the sum of squares, which can be used in a F-statistic). GLM generalizes the linear model used in ANOVA by allowing any other type of distribution of the residuals (and optimizes the likelihood function, which only allows a t-test based on an estimated error of the coefficients). Jul 6, 2017 at 23:36 • So an anova is an Glm, but a Glm is not only anovas. Jul 6, 2017 at 23:56 • So an ANOVA is GLM. It is a bit semantic now. 1) GLM includes ordinary least squares. You can't really say OLS is a GLM (since GLM is general and OLS is not). Better would be to say that OLS is GLM with identity link and variance functions. 2) ANOVA is not exactly the same as OLS. It is the analysis performed on a linear regression (on the sum of squares of residuals). Also, ANOVA refers to analysis of reduced var_residuals which you can't do (makes no sense) with GLM. However, in practice, people doing ANOVA often do OLS and model effects. But technically$ANOVA \neq OLS \in GLM\$, Jul 7, 2017 at 1:56
• The normal linear model is a special case of GLM, while OLS is a distribution free algorithm to find its solution. The classic t-test is a special case of the normal linear model. Jul 7, 2017 at 5:58
• Whoa! For some people GLM is general linear model and for some is GLM is generalized linear model, not all identical. This thread hasn't avoided confusion. Feb 22, 2022 at 18:28

This is a common point of confusion, as the word ANOVA is used with different meanings in different textbooks / software packages. I'll try to sort this out a bit:

Historically, ANOVA is a method to partition out the contribution of different factors to the variation in a continuous variable. The classical ANOVA measured this contribution via the sum of squares, which corresponds to the assumptions of an lm (iid normal and so on), thus you will often read that AVOVA = normal distribution. This also implies that it doesn't matter if you calculate an ANOVA directly, or first an lm and then perform the ANOVA on the fitted lm, it's basically the same model.

This obviously doesn't generalise to GLM(Ms), but people would still like to test for the significance (and contribution) of factors or factor groups in those models. Thus, the ANOVA concept has been extended, and a more modern way to look at ANOVA is that an ANOVA is just a series of tests that add / remove predictors or predictor groups in a regression to test for their overall significance, as well as changes in some metric of fit (pseudo R2, there are multiple definitions, most based on deviance).

So, regardless of lm, glm, glmer, if you have a model with a predictor color = red, green, blue, in R

m1 <- lm/glm/glmer(res ~ color)
summary(m1)


will give you p-values for the contrasts between red, green, blue, while

am1 <- anova(m1)
summary(am1)


will essentially do a (likelihood ratio) test to decide if the predictor color improves the fit significantly (note that adding color, you are adding 2 df / parameters at once), and it will also provide a feedback about the improvement of fit.

In R there are different ANOVA functions (aov, anova, car::ANOVA) that slightly differ in their use and appropriateness for particular regressions and questions. car::ANOVA is very versatile and allows changing between type II, II ANOVA (explanation here)

Note that you can also use the anova command in R to do a LRT between two models, as in anova(m1, m2), so in a way you can see anova(m1) simply as a shorthand to compare m1 all of its smaller submodels via a LRTs.

regression & ANOVA are General Linear Models - for Normally distributed data. Example of Generalized Linear Models can be Logistic Regression that uses logistic-function, or logistic-curve (S-shaped), that converts log-odds to probability of belonging to certain class (in binary classification) or other non-linear functions assigned in arguments of API's glmm-implementation

Also, I know that it's possible to add mixed effects in a GLMM. Is it the case for ANOVAs?

yes ANOVA can be GLMM for analysing within-group & between-group variability e.g. having several experiments repeated (as random effect) & compaaring it to the group after any treatment (as fixed effect)