2
$\begingroup$

Here is somewhat simplified structure of the data I have, since fixed effects are quite straight forward, however, random effects are giving me a headache (like I said something new :) ):

studentId   | courseId | courseName | year | courseGroup | timespent | mark
stud1       | 19       | M101       | 2008 | F           | 12.3      | 3.7
stud1       | 21       | E102       | 2008 | C           | 2.3       | 4
stud1       | 109      | H300       | 2008 | E           | 22.3      | 3
stud2       | 19       | M101       | 2008 | F           | 3.3       | 3
stud2       | 21       | E102       | 2008 | C           | 12.3      | 3.3
stud3       | 200      | M101       | 2009 | F           | 12.3      | 3.7

There are 2000 observations for around 300 students within several courses (3-10 courses per student), over 4 years. Each course belongs to one of the groups (F, C, or E). idcourse is a specific offering of a course - for example, stud1 and 2 took the same course M101 during 2008, while stud3 took the same course in different offering (2009, id-200).

The idea is to predict the final grade (mark) based on the time spent reading course materials (for example). This means that timespent would be a fixed effect. But, I'd like to examine the effect of different courses within the various groups. That is, how the predictive power of time depends on the course group, and maybe course or even course offer.

I was trying to fit a null model with different settings, so far the best model I was able to fit is this

lmer(mark~(1|studentId)+(1|courseId)+(1|courseName:courseGroup), data=data_final)

but I think the model I wanted is this

lmer(mark~(1|studentId)+(courseId|courseName/courseGroup), data=data_final)

which gives the following error:

Error: number of observations (=2017) <= number of random effects (=6355) for term (courseId | courseGroup:courseName); the random-effects parameters and the residual variance (or scale parameter) are probably unidentifiable

I understand the error, but I'm not sure I quite understand the difference between these two models. I mean, I do understand what those models mean, but which one would be correct in this case? (If we ignore the fact that I wasn't able to fit the second one...)

$\endgroup$
  • $\begingroup$ small detail, but are you switching from idcourse to courseId between your data description and your model description? $\endgroup$ – Ben Bolker Dec 28 '14 at 23:28
3
$\begingroup$

I think you want

lmer(mark~courseGroup+(1|studentId)+(1|courseName/courseId),
     data=data_final)
  • courseGroup seems as though it might conceptually be fixed rather than random. Even if it's conceptually a random effect (= chosen from a possibly-hypothetical larger population of course groups), it doesn't practically work well to fit random effects if there are few (e.g. <5) levels of the grouping variable. As long as courseName has unique levels (i.e. you don't have the same courseName in different groups referring to different courses), you don't have to worry about nesting: if you did, you would use : to indicate the interaction; I would use (1|courseGroup:courseName)+(1|courseGroup:courseName:courseId) to be explicit, although I would check whether (1|courseGroup:(courseName/courseId)) gave the same result (it should, in principle).

  • the a/b syntax denotes b nested within a.

  • this allows for variability among course groups, among course names within course groups, and among course IDs within groups (although technically it doesn't estimate variability among course groups).
  • you might want to add timespent to the left of the bar for some grouping factors, e.g. timespent|courseName/courseId or (equivalently timespent:courseGroup) to see if the effects of time spent vary across levels, although you have to proceed cautiously here since it's pretty easy to overwhelm your data this way (check Barr et al 2013 "keep it maximal" and the counterargument by Shravan Vasishth and make up your own mind ...)
$\endgroup$
  • $\begingroup$ Thanks Ben, really useful explanation. CourseGroup should definitely be a fixed effect, I was trying to fit course within the courseGroup as a random. I thought I read somewhere that a/b means a nested within b? Guess my mistake... $\endgroup$ – Srecko Dec 28 '14 at 23:51
  • $\begingroup$ I find your comment in the 1st bullet point ("... doesn't practically work well to fit random effects if there are few... levels...") interesting. Is the "practically" meaning this is something about the way its implemented in the underlying code in R, or is it something about the logic of mixed-effects models? (This may be vaguely related to a question I had a while ago: Fixed effect vs random effect when all possibilities are included in a mixed effects model.) $\endgroup$ – gung - Reinstate Monica Dec 29 '14 at 0:19
  • 2
    $\begingroup$ It's more fundamental than the implementation in R. I don't know of a really good discussion; maybe it would make a good follow-up CV question (haven't looked yet to see what's here already). Googling mixed models "small number of levels" does give some interesting results, although e.g. faculty.ucr.edu/~hanneman/linear_models/c4.html (which otherwise looks very good) confounds "small number of levels" with "all possibilities are included", which are really separate issues. See also rpubs.com/bbolker/4187 ... $\endgroup$ – Ben Bolker Dec 29 '14 at 0:44
  • $\begingroup$ @Ben, There is one more thing I'm not sure about... everything seems to be fine when I try to fit the model as you suggested. However, adding interaction between courseGroup and timespent gives me a warning: fixed-effect model matrix is rank deficient so dropping 1 column / coefficient. Any ideas why that might be the case? Thanks! $\endgroup$ – Srecko Dec 29 '14 at 1:44
  • $\begingroup$ It means that some set of your fixed effects are multicollinear: see e.g. stackoverflow.com/questions/26449969/… $\endgroup$ – Ben Bolker Dec 29 '14 at 1:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.