Removing Subject Random Effect From Mixed-Effects Models & Violation of Independence

Question

I'm running logistic mixed-effects models for a project using glmer(), but ran into a few problems with model fit.

In this model, there are 2 fixed effects:

1. Factor A, a continuous variable (but only with 3 values, -25, 0, and 25)
2. Factor B, a categorical variable with 4 levels

The only random effect structure in the model is (1|subject).

I collected data from 2 different samples for this project. For one of the samples that used this model structure I ended up getting a singularity warning, and the random effect/standard deviation is 0. The model ran just fine for the sample, but the random effect was relatively small - when I removed the random effect and ran a regular logistic regression using this second sample, and compared the regular logistic regression vs mixed effects logistic model using anova.Mermod(), the addition of the random effect didn't seem to contribute significantly to the glmer model with the random effect. Removing the random effect structure for models that previously ran into singularity issues also seemed to fix the problem (and it didn't give me weird coefficient estimates like in previous cases).

So this all seems good, except that removing the random effect of subject would also violate assumptions of independence (even though subject doesn't seem to contribute a significant amount of variance), and is it okay to violate this assumption in this particular scenario? If not, what would be some alternative solutions to this problem (the only thing I can think of is to run a mixed-ANOVA, treating Factor A as 3 levels of a discrete factor, and then following up with linear contrasts to look at the effect of Factor A at each level of Factor B). Any advice would be much appreciated!

---

EDIT: I ended up running GEEs to get around this issue, since the goal was not to model subject as a random factor, but rather to account for the within-subject variance.

Answer 1

I collected data from 2 different samples for this project.

Is there a reason to model the two samples completely separately? You could model them together and assume the random-effects variances were the same in each sample, e.g.

glmer(response ~ A*B*sample + (1|subject), ...)

? This will fit separate coefficients for A, and B, and their interactions for each sample. You would need to decide how to parameterize the model: if you want completely separate coefficients for each sample, you'd use ~(A*B):sample+sample-1, e.g.

dd <- expand.grid(A=c(-25,0,25),B=factor(letters[1:4]),sample=factor(1:2))
colnames(model.matrix(~(A*B):sample+sample-1, data=dd))

If you wanted instead to parameterize by mean effect and deviation between mean and sample 1, you'd use sum-to-zero contrasts: ~A*B*sample with contrasts=list(sample=contr.sum)

For one of the samples that used this model structure I ended up getting a singularity warning, and the random effect/standard deviation is 0. The model ran just fine for the [other] sample, but the random effect was relatively small - when I removed the random effect and ran a regular logistic regression using this second sample, and compared the regular logistic regression vs mixed effects logistic model using anova.Mermod(), the addition of the random effect didn't seem to contribute significantly to the glmer model with the random effect. Removing the random effect structure for models that previously ran into singularity issues also seemed to fix the problem (and it didn't give me weird coefficient estimates like in previous cases).

In general a model with a singular fit (random effects variance = 0) for a single random effect grouping will give exactly the same results as the model with the random effect removed. If the random-effects variance is small (in magnitude, not necessarily significant/non-significant!) then the difference will be small but not zero.

So this all seems good, except that removing the random effect of subject would also violate assumptions of independence (even though subject doesn't seem to contribute a significant amount of variance), and is it okay to violate this assumption in this particular scenario?

Opinions differ widely on this (see e.g. the GLMM FAQ section on singular fits. I personally prefer not to drop random-effects terms, just because they're small or non-significant, especially when they're part of the experimental design.

If not, what would be some alternative solutions to this problem

I would probably just report the results from the singular fit. A singular fit is not necessarily wrong; it just suggests that the random effect may be small and/or poorly constrained by the data. I would prefer to analyze the two samples together (see above), but I don't see that it's required to drop the random effect from the second sample for consistency.

See the link above for lots of other possibilities (e.g. using a Bayesian prior to constrain the random-effects variance away from zero, as in the blme package).

Answer 2

After receiving some helpful feedback from folks from methods discussion group, I was informed that generalized estimation equations would potentially get around this issue (and it did after I ran the model). For this particular project, subject was largely a nuisance variable, and the initial intention for using mixed-models was to account for the within-subject variance/non-independence of observation. Since we didn't actually need to model the random effect of subject (in that we didn't need to obtain estimation/information about this variable), GEEs seemed better-fitted for the job anyways. Running GEEs also solved some of the other issues I was getting (i.e., getting estimates that were off by several magnitudes and would not have made sense in the context of the current experiment).