How to model nested data with repeated measures

Question

My data is like this: I have N students from different courses (variable course_id) that where asked about household income (variable income) and had to take part in two math tests; one test was considered "income fair" and the other was not (variable test_type). The results of the math test are in the variable test_results. Each student took both of the tests. And students are grouped in courses. Students have an ID variable student_id.

My general question is whether the effect of the income on test results varies depending on the test type, i.e. is smaller for the income fair test. My challenge is now to model this because students are nested in courses (each course has multiple students; each student in just one course) and there is repeated measures (each student takes two tests: the fair and the normal test). I want to take into account that there can be a random intercept and slope of income on test results for for the courses. How to do this?

I've read and tried so many thing like this and am quite confused now.. What I do in R is this:

my_fit <- lme4::lmer(test_results ~ income + (1|test_type) + (1|course_id) +
                      (income|course_id), data= anna_long)
summary(my_fit)

But I am not sure whether this is correct. And there is no p value for the income predictor... I am lost..

---

Before I also wanted to take into account that there can also be a random intercept and slope for each student and what I did was...

my_fit <- lme4::lmer(test_results~ income + (1|test_type) + 
    (1|course_id/student_id) + (income|course_id/student_id), 
    data= anna_long)
summary(my_fit)

... but this resulted in the error

number of observations (=361) <= number of random effects (=362)

... Therefore I discarded the random effects associated with single students and focused on random effects of courses as shown above.

Answer 1

A few things here:

• it probably doesn't make sense to treat test_type as a random effect. Test type may be a nuisance variable (i.e., you're not primarily interested in the population-level effect of test type), but (1) the levels (income-fair or not) aren't really exchangeable and (2) it's impractical to estimate a variance from only two observations (which is effectively what you would be trying to do).
• "students are nested in courses (each course has multiple students; each student in just one course)": this means you would normally want a random-effect grouping variable of the form course/student ... but ...
• presumably income varies only across students within courses, not across tests within students, so you can't estimate variation in income effects among students
• you might be interested in variation among students within courses in the difference between test types ((1+test_type|course:student)), but because you only have two observations per student, this effect is confounded with the residual variance term, leading to the error you saw. (There are ways around this if you really want ...)

So I think you want:

test_results ~ income*test_type + (income*test_type|course) + (1|course:student)

where course:student is effectively "student within course". The first term adds variation among courses in the slopes and intercepts (i.e., expected difference in test results for the two test types at a reference value of income; you might want to center your income variable); the second adds a random intercept term for students within courses.

• lme4 doesn't give p-values for Reasons, but if you load the lmerTest package instead of/in addition to lme4, before fitting your model, the summary() output should give you p-values.