Finding the optimal structure of fixed / random effects in a 3 way full factorial repeated measures design with nlme/lme4

I’m looking for help in finding the best / optimal structure of fixed and random effects in my repeated measures experiment data along with an explanation on why do I choose one solution over the other.

By reading various books I found a chapter on Repeated Measures designs by Andy Field (Discovering Statistics using R). He uses nlme and suggests to set up a baseline model with the following formula:

baseline<-lme(attitude ~ 1, random = ~1|participant/drink/imagery, data = longAttitude, method = "ML")


from this point he advices to add fixed effects using the update function and finding the best fit (by comparing AICs or logLik).

Following his exemplary data, above formula would mean (as I understand it) a random intercept of each participant with two repeated measurement conditions (drink and imagery nested within each participant).

On the other hand, @henrik in his post Order of nested random effects in lme4, URL (version: 2017-04-13) states that

“As all participants did see all levels of your factor f1, the factor is not nested within id. A factor is nested within another factor if each instantiation of the higher order factor does not see all instantiations of the lower order factor (e.g., a factor is nested within id if id1 saw levels A and B, but not levels C and D, whereas id2 saw C & D but not levels A & B).”

Given those two examples I’m confused on how should I approach my data so that I won’t introduced any major errors while fitting my models.

My experiment was investigating the reaction times of Yes/No judgments regarding two different targets (self vs other) on two different aspects (feelings vs behavior) in two different time perspectives (recent vs distant).

Each of the 3 factors (target, aspect, time) was within subjects with fully counterbalanced order between subjects.

Also, each participant made a total of 128 judgements (on a Yes / No scale) – number of trials breaks down to 16 repetitions for every 2x2x2 (target * mode * time) combination. Each question (judgment) addressed a unique trait (happy, sad etc), but traits itself are not interesting to me.

The important part is that for every combination of my within subject factors there is a total of 16 judgments and that the depended variable is latency of the judgment controlled all 3 experimental factors + response type (was it Yes or was it No? – yes judgments are generally faster and I’d like to account for that) and trial number (responses become faster as the procedure advances and I’d like to control that too).

In previous research I used a regular GLM in SPSS to analyze similar data. Here I’ve been advised by a colleague reviewer to shift toward multilevel models (HLM, MLM, LMEM) as this can help me with:

1. Controlling for response type (Yes vs No) which would give a lot of missing cells when collapsed across all factors + response type
2. Controlling for the fact, that extremely low latencies (<200msec) are considered invalid and dropped from the analysis anyway.

Eventually I’d like to report anova-like type III F’s (or Chi2) of all main effects and their interactions along with the possibility to test for simple main effects equivalents if a 3-way interaction comes up to be significant.

To simulate data for 100 (in reality I have data for 244 subjects) participants representing a structure I will work with I came up with this code:

library(AlgDesign) #for generating a factorial design)
df <-gen.factorial(c(16,2,2,2,100), factors = "all", varNames = c("rep", "A", "B", "C", "Subject"))
df$rep <- as.numeric(df$rep)
df$Subject <- as.numeric(df$Subject)
logRT <- rnorm(n=12800, m=7, sd=1) # outcome variable
trialno<- rep(1:128, times = 100)
response<- rbinom(12800, size = 1, prob = c(0.3, 0.7))
df <- cbind(df, logRT, trialno, response)


Using aField's approach I should startwith a “baseline” model lme(logRT ~ 1, random = ~1|Subject/A/B/C) and work on fixed effects (which I'd like to be full factorial design):

1. Add full factorial fixed effects (ABC)
2. Add response as factor to the fixed part

On the other hand, following @Henrik post, this random structure is simply wrong.

Also I noticed that it makes a difference (for logLik) the order of nested factors - results from Subject/A/B/C differ from results of Subject/B/A/C syntax and I don't understand why and what solution is better for my experiment.

One other problem I came up is that I cannot freerly experiment with random sturctures as models are not converging. For example using my real data computing a model formula of lme(logRT ~ A * B * C + trialno, random = trialno|Subject/A/B/C) (representing a full factorial fixed effcts with random slope of trail number for every Subcjent) takes ages and I feel that it might be overspecified.

If any of the above make no sense and there is a established protocol that I might replicate and cite in the article I’d love to get a hint.

Otherwise any comments or full answers on best practices with my data structure are really more than welcome.

Thank you very much.