What are the differences between ANOVAs and GLMs?

Question

I'm wondering how different are generalized linear model and ANOVA. I thought it was because of categorical vs continuous variables, but it doesn't seems to be the case.

Also, I know that it's possible to add mixed effects in a GLMM. Is it the case for ANOVAs?

So, why would one use GLM vs ANOVAs?

Here is an example:

# Difference regression vs ANOVA vs GLM
gr1 = rnorm(10,10,1)
gr2 = rnorm(10,20,1)

df=data.frame(nm = c(gr1,gr2), 
              gr = c(rep(0,length(gr1)),
                     rep(1,length(gr2))))

ANOVA

summary(aov(nm~gr, data = df))
    
            Df Sum Sq Mean Sq F value  Pr(>F)    
gr           1  496.4   496.4   621.1 2.1e-15 ***
Residuals   18   14.4     0.8                    
---
Signif. codes:  
0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

GLM

glm(nm~gr,family = "gaussian", data = df)

Call:  glm(formula = nm ~ gr, family = "gaussian", data = df)

Coefficients:
(Intercept)           gr  
     10.050        9.964  

Degrees of Freedom: 19 Total (i.e. Null);  18 Residual
Null Deviance:      510.8 
Residual Deviance: 14.39    AIC: 56.17

And once we are there, why not comparing linear regressions and t-test!

Linear regression

summary(lm(nm~gr, data = df))

Residuals:
     Min       1Q   Median       3Q      Max 
-1.13990 -0.85335  0.00061  0.41865  1.92899 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  10.0502     0.2827   35.55  < 2e-16 ***
gr            9.9638     0.3998   24.92  2.1e-15 ***
---
Signif. codes:  
0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.894 on 18 degrees of freedom
Multiple R-squared:  0.9718,    Adjusted R-squared:  0.9703 
F-statistic: 621.1 on 1 and 18 DF,  p-value: 2.095e-15

t-test

tt=t.test(df$nm~df$gr);tt 
    Welch Two Sample t-test

data:  df$nm by df$gr
t = -24.922, df = 17.914, p-value = 2.347e-15
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -10.804012  -9.123526
sample estimates:
mean in group 0 mean in group 1 
       10.05023        20.01400


(tt$statistic)^2
       t 
621.0907 

Answer 1

This is a common point of confusion, as the word ANOVA is used with different meanings in different textbooks / software packages. I'll try to sort this out a bit:

Historically, ANOVA is a method to partition out the contribution of different factors to the variation in a continuous variable. The classical ANOVA measured this contribution via the sum of squares, which corresponds to the assumptions of an lm (iid normal and so on), thus you will often read that AVOVA = normal distribution. This also implies that it doesn't matter if you calculate an ANOVA directly, or first an lm and then perform the ANOVA on the fitted lm, it's basically the same model.

This obviously doesn't generalise to GLM(Ms), but people would still like to test for the significance (and contribution) of factors or factor groups in those models. Thus, the ANOVA concept has been extended, and a more modern way to look at ANOVA is that an ANOVA is just a series of tests that add / remove predictors or predictor groups in a regression to test for their overall significance, as well as changes in some metric of fit (pseudo R2, there are multiple definitions, most based on deviance).

So, regardless of lm, glm, glmer, if you have a model with a predictor color = red, green, blue, in R

m1 <- lm/glm/glmer(res ~ color)
summary(m1)

will give you p-values for the contrasts between red, green, blue, while

am1 <- anova(m1)
summary(am1)

will essentially do a (likelihood ratio) test to decide if the predictor color improves the fit significantly (note that adding color, you are adding 2 df / parameters at once), and it will also provide a feedback about the improvement of fit.

In R there are different ANOVA functions (aov, anova, car::ANOVA) that slightly differ in their use and appropriateness for particular regressions and questions. car::ANOVA is very versatile and allows changing between type II, II ANOVA (explanation here)

Note that you can also use the anova command in R to do a LRT between two models, as in anova(m1, m2), so in a way you can see anova(m1) simply as a shorthand to compare m1 all of its smaller submodels via a LRTs.

Answer 2

regression & ANOVA are General Linear Models - for Normally distributed data. Example of Generalized Linear Models can be Logistic Regression that uses logistic-function, or logistic-curve (S-shaped), that converts log-odds to probability of belonging to certain class (in binary classification) or other non-linear functions assigned in arguments of API's glmm-implementation

Also, I know that it's possible to add mixed effects in a GLMM. Is it the case for ANOVAs?

yes ANOVA can be GLMM for analysing within-group & between-group variability e.g. having several experiments repeated (as random effect) & compaaring it to the group after any treatment (as fixed effect)