# Fixed or random effect in linear mixed effect

Let say I've a dataframe as :

image_id image_group group x y nn


I've 20 images defined by image_id ; each image belongs to a group G1 or G2 defined in the image_group column. Eeach image is a list of points from different groups (e.g. A, B and C) defined by their coordinates x and y. nn represents the number of neighbour of the point in a predefined radius e.g. r=100.

My goal is to compare two groups of points by their minimal distances. As an example A vs B : I compute for each point of A : the distance to all points of B and report the minimal distance. Thus if I've 100 A points it will report 100 distances.

Using these distances I want to compare group G1 vs G2 and see if A vs B distances are different.

My idea was to use a linear mixed model and use nn as a covariate but not sure if it should be used as fixed or random effect.

In R (for the example I generate a dist column to simulate the distance between points. Here group G1 has a lower number of neihgbour compared to G2.

require(lme4)
require(lmerTest)

set.seed(123)
dat <-
data.frame(
image_id = c(rep(1:5,1000),rep(6:10,1000)),
image_group = c(rep("G1",1000),rep("G2",1000)),
dist = rnorm(2000,mean=50,sd=10),
nn = c( round(rnorm(1000,mean = 20,sd=5)),round(rnorm(1000,mean=30,sd=5))))

# version without using nn as covariate
mixed.lmer <- lmer(dist ~  image_group + (1|image_id), data = dat)

# version with nn as random effect
mixed.lmer2 <- lmer(dist ~  image_group + (1|image_id) + (1|nn), data = dat)

# version with nn as fixed effect
mixed.lmer3 <- lmer(dist ~  image_group + nn + (1|image_id) , data = dat)

anova(mixed.lmer)
anova(mixed.lmer2)
anova(mixed.lmer3)


Hereby the results

> anova(mixed.lmer)
Type III Analysis of Variance Table with Satterthwaite's method
Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
image_group 173.41  173.41     1  9998   1.733 0.1881

> anova(mixed.lmer2)
Type III Analysis of Variance Table with Satterthwaite's method
Sum Sq Mean Sq NumDF  DenDF F value Pr(>F)
image_group 90.472  90.472     1 1793.8  0.9183 0.3381

> anova(mixed.lmer3)
Type III Analysis of Variance Table with Satterthwaite's method
Sum Sq Mean Sq NumDF DenDF F value  Pr(>F)
image_group 385.83  385.83     1  9997  3.8563 0.04959 *
nn          213.12  213.12     1  9997  2.1300 0.14447
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1


Which one will be the most suitable ? In this simulated example, results can be really different depending on the model. As number of neighbours can obviously bias the minimal distance calculation (More points in the vicinity -> more chance to have a point of group B close )

Thank you the help and comments

EDIT

Following @Doctor Milt advice I used a gam

Here's the plot of the model

model4 <- gam(dmin ~  image_group + s(nn) + s(image_id, bs = "re"), data = dat)
plot(model4)


• Isn't nn a numeric variable? In mixed.lmer2 you're treating it as a categorical variable, which doesn't make sense to me. Commented Apr 28, 2023 at 11:38
• Your argument about the number of neighbours having an effect on the minimal distance is sensible, and I can totally believe that the relationship between the two is not linear (which is the assumption in mixed.lmer3), so maybe try a spline? Commented Apr 28, 2023 at 11:40
• @DoctorMilt indeed it's not categorical. And indeed thinking about it has no sense to use it as a random effect then. So model 2 should be thrown away. Thanks for the comment Commented Apr 28, 2023 at 11:49
• Regarding the spline. Which package will you use. Sorry I'm not an expert in modelling :/ Commented Apr 28, 2023 at 11:50
• No problem! I've added a proper answer. Commented Apr 28, 2023 at 11:53

Deciding whether to include a covariate using fixed or random effects is relevant when the variable is categorical. In this situation, nn is numeric (an integer).

For example, here are the estimates of the random effects and their standard errors from your second model (mixed.lmer2). Clearly not what we want!

As I mentioned in my comment, your argument about the number of neighbours having an effect on the minimal distance is sensible, though the relationship between the two might be nonlinear. You could use the mgcv package to fit a GAM where you include nn as a spline.

mgcv allows you to add random effects, so you could keep image_id in there. The syntax would be something like

library(mgcv)
model4 <- gam(dist ~  image_group + s(nn) + s(image_id, bs = "re") , data = dat)

• Thank you for the great answer. Regarding the relationship between distance and number of neighbours, what will be the best way to test it ? gam and lmer gave very different results on the test dataset I gave in example. I also tested in my real dataset and results are also quiet different. Thanks Commented Apr 28, 2023 at 12:39
• Did you try plotting this model? What does the spline look like? Commented Apr 28, 2023 at 13:15
• I edited my question to add the plot Commented Apr 28, 2023 at 13:33
• Looking at the plot on the left (s(nn) vs. nn), I'm not surprised you get very different results. In mixed.lmer3 you're assuming that the relationship between the outcome and nn is linear, but there's clear evidence of nonlinearity. Once nn exceeds 250 or so, it looks flat to me. Commented Apr 28, 2023 at 13:52
• ok thus I should go for the gam method then I guess ? Commented Apr 28, 2023 at 13:55