# Understanding fixed versus random effect

I am trying to get my head around the difference between a fixed effect versus a random effect. To do this, I am looking at the Student GPA example seen here (from m-clark.github). This example is "assessing the factors that predict college grade point average (GPA). Each of 200 students are assessed for six occasions (each semester for the first three years), so we have observations clustered within students."

The example starts by fitting a standard linear regression, with only occasion as a fixed effect.

load('data/gpa.RData')
gpa_lm = lm(gpa ~ occasion, data = gpa)
summary(gpa_lm)


This produces the following output:

Call:
lm(formula = gpa ~ occasion, data = gpa)

Residuals:
Min       1Q   Median       3Q      Max
-0.90553 -0.22447 -0.01184  0.26921  1.19447

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 2.599214   0.017846  145.65   <2e-16 ***
occasion    0.106314   0.005894   18.04   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.3487 on 1198 degrees of freedom
Multiple R-squared:  0.2136,    Adjusted R-squared:  0.2129
F-statistic: 325.3 on 1 and 1198 DF,  p-value: < 2.2e-16


Subsequently, the example demonstrates the fitting of a mixed effects model, where occasion is again included as a fixed effect, but this time student is included as a random (intercept) effect:

library(lme4)
gpa_mixed = lmer(gpa ~ occasion + (1 | student), data = gpa)
summary(gpa_mixed)


Producing the following output:

Linear mixed model fit by REML ['lmerMod']
Formula: gpa ~ occasion + (1 | student)
Data: gpa

REML criterion at convergence: 408.9

Scaled residuals:
Min      1Q  Median      3Q     Max
-3.6169 -0.6373 -0.0004  0.6361  2.8310

Random effects:
Groups   Name        Variance Std.Dev.
student  (Intercept) 0.06372  0.2524
Residual             0.05809  0.2410
Number of obs: 1200, groups:  student, 200

Fixed effects:
Estimate Std. Error t value
(Intercept) 2.599214   0.021696   119.8
occasion    0.106314   0.004074    26.1

Correlation of Fixed Effects:
(Intr)
occasion -0.469


Now I can clearly see that in both models, the estimate for the global intercept remains the same (2.599). And I understand that in the mixed effects model each student gets their own intercept estimate; I can see these by running the following code:

coef(gpa_mixed)$student[1:5,]  Resulting in the following output (only the first 5 students shown):  (Intercept) occasion 1 2.528319 0.1063143 2 2.383636 0.1063143 3 2.687471 0.1063143 4 2.412573 0.1063143 5 2.629598 0.1063143  My question, hence, is what is the difference between the above mixed effects model, which includes student as a random effect, versus a fixed effects model that includes student as a fixed effect? In both cases an intercept effect is estimated for each student, so what is it about the two models that is different? To exemplify, here is the fixed effects model: gpa_fixed = lm(gpa ~ occasion + student, data = gpa) summary(gpa_fixed)  This model produces the following summary – note that again only students 1-5 are shown, with the estimate for student 1 being represented by "(Intercept)": Call: lm(formula = gpa ~ occasion + student, data = gpa) Residuals: Min 1Q Median 3Q Max -0.8676 -0.1474 0.0072 0.1438 0.7325 Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 2.518e+00 9.892e-02 25.450 < 2e-16 *** occasion 1.063e-01 4.074e-03 26.096 < 2e-16 *** student2 -1.667e-01 1.392e-01 -1.198 0.231301 student3 1.833e-01 1.392e-01 1.318 0.187967 student4 -1.333e-01 1.392e-01 -0.958 0.338196 student5 1.167e-01 1.392e-01 0.838 0.401995  From this I can see that the estimate for the slope effect of occasion remains unchanged (0.10634 in both the fixed effects and the mixed effects model), but the estimates for the intercept effect for each student are slightly different. For example, in the mixed effects model, student 1 has an intercept estimate of 2.528319, whereas in the fixed effects model the intercept estimate for student 1 is 2.518. Similarly, in the mixed effects model, student 2 has an intercept estimate of 2.383636, whereas in the fixed effects model the intercept estimate for student 2 is 2.518-0.1667=2.3513. Admittedly these values are very similar, but they are not identical. Are these values different only because of the way that they are estimated? i.e. partial pooling/shrinkage in the case of the mixed effects model versus least-squares in the fixed effects model? My understanding is that when you include student as a random effect, the estimate for each student's intercept is drawn from a normal distribution centered on the global intercept estimate. Whereas when you include student as a fixed effect, each student's intercept can be estimated with whatever value is best (i.e. best according to least squares) without any constraint as to what distribution that estimate is drawn from. Is my understanding here correct? Why would it matter, from a statistical point of view, if I modeled student as a fixed effect, rather than as a random effect? Is it for example because with the fixed effect you cannot "borrow strength" from other groups? And because with the fixed effect you are having to estimate a greater number parameters than with the random effect, which only considers variance around the global intercept estimate? Thank you in advance! Edit: I also have another question - how exactly do you interpret the value of the random effect? Random effects: Groups Name Variance Std.Dev. student (Intercept) 0.06372 0.2524 Residual 0.05809 0.2410 Number of obs: 1200, groups: student, 200  The value for the variance effect size is 0.06372. Does this mean that, when variance of all the individual student intercept estimates is 0.06372? I tried that using var(ranef(gpa_mixed)$student) but this returns a value of 0.0553. If anyone could shed some light on how to intuitively interpret the random effect estimates, that would really help!

This may require a long explanation but I can point you towards one blog post that has references to the relevant books. In a nutshell, if fitted for just numerical reasons, then a random-effects-model is useful when sample sizes are small, as information sharing and shrinkage happens - less overfitting and all that. If sample sizes are large then it does not make any practical difference in most cases when one considers the added complexity of the model.

From a conceptual or philosophical view point, a random-effect model can be used to estimate 'super-population' parameters.

GELMAN, A., & HILL, J. (2007). Data analysis using regression and multilevel/hierarchical models. Cambridge, Cambridge University Press.

lme4: Mixed-effects modelling with R. Douglas M. Bates (2010)

Kruschke, J. K. (2014). Doing Bayesian data analysis: A tutorial with R, JAGS, and Stan, second edition.

https://laplacebayes.wordpress.com/2019/01/04/partial-complete-or-no-pooling-information-content-sample-sizes-and-shrinkage-in-multilevel-regression-models/

I hope this is useful.

Cheers