2
$\begingroup$

Structure of my data

I have been trying to find out the most adequate formula for my data but I found no example that reflects the structure of my data, as pictured in figure above.

My data is dichotomous [correct/incorrect (0,1)] for vocabulary tests:

So here is what we have:

Subjects

Response as DV

Time:Pretest,Posttest

Group: Control, View, NoView

Word: 20 items tested

7 additional fixed effects

I want to test the significance of change over the 2 time points.

Problem: I want to make sure that comparison analysis for groups is based on individual score change on each 20 item rather than taking individual or group average.

Potential Model

mod1 <- glmer(Response ~ Time*Group + (Time| Group: Subjects)
+ (Time| Group) + 
+ (Time|Subject) 
+ (Time|Word), 
data= vocabDat, family='binomial')

Question

  1. How far is my formula correct and reflects my aims?
  2. Is it true that Rasch Model should be more useful for my data?
$\endgroup$
4
$\begingroup$

I think you would neeed to provide more details in your question to enable us to provide a more suitable answer. In the meantime, please see below.

The formula for your potential model mod1 suggests that you are treating Group as a random grouping factor. But Group has only 3 levels - Control, View, No View - and those levels are likely the only ones you are interested in. In other words, you are not considering that these levels are representative of a larger set of levels - you just care about the 3 concrete levels Control, View and No View. This suggests that you should allow Group to appear in the fixed effects portion of your model, not in the random effects portion. In particular, this implies that you shouldn't have terms like (Time|Group) or (Time|Group:Subject) in your model.

Unlike Group, Subject and Item are genuine random grouping factors in that:

  • The subjects you included in your study are representative of a larger set of subjects to which you wish to generalize the study findings;

  • The items you included in your study are representative of a larger set of items to which you wish to generalize the study findings.

Ideally, the subjects included in the study would have been selected at random from the larger set of subjects and the items included in the study would have been selected at random from the larger set of items. This would ensure their representativeness.

From the diagram you provided, it seems that your two random grouping factors, Subject and Item are fully crossed (?). If this is indeed the case, each subject included in the study is exposed to each of the items included in the study, so it would make sense to allow for terms such as (...|Subject) and (...|Item) in your model formula.

As an aside, other possibility in terms of study design would be that each subject included in the study is exposed to some but not all of the items included in the study - with some overlap between subjects in terms of the items they are exposed to, in which case Subject and Item would be partially crossed. Yet another possibility would be that each subject is exposed to totally different items compared to all other subjects (so that Item is nested within Subject). Let's ignore these possibilities for now, as they don't seem to match your own study design.

The part that is not clear from your question is what exactly was measured for each subject by item combination in your study? Specifically, how often was your Response measured for each subject by item combination?

Were your subjects allocated to one of 3 different groups - Control, View or No View - and then exposed in turns to each of the items included in the study? Was their Response assessed both before the exposure (Pretest) and after the exposure (Postest)? In that situation, it makes sense to allow the effect of Time to vary at random across subjects and items but not the effect of Group. So your preliminary model could be something like:

mod1 <- glmer(Response ~ Time*Group +
                         (Time|Subject) +
                         (Time|Item), 
              data = vocabDat, 
              family = 'binomial')

Just like Andre, I can't comment on the Rasch Model as I am not familiar with it.

Addendum:

Let's say your study design is much simpler (you only have one item) and your model is much simpler:

mod1 <- glmer(Response ~ Time*Group + (1|Subject),
              data = vocabDat, 
              family = 'binomial')

The model equation would be:

LOi = (beta0 + S0i) + beta1*TimeDummy + 
     beta2*ViewDummy+ beta3*NoViewDummy + 
     beta4*TimeDummy*ViewDummy+ beta5*TimeDummy*NoViewDummy

where

  • LOi = log odds of correct response on sole item for i-th subject;
  • S0i is a random intercept (aka random subject effect);
  • TimeDummy equals 1 when Time = Posttest and 0 when Time = Pretest;
  • ViewDummy equals 1 when Group = View and 0 else;
  • NoViewDummy equals 1 when Group = NoView and 0 else.

This model will ultimately enable you to compare the odds of a correct response between posttest and pretest, separately for each group, while allowing for subject-to-subject variability in responses.

Now, if you revert back to your own design and account for item-to-item variability as well, a simplified model may be:

mod1 <- glmer(Response ~ Time*Group + (1|Subject) + (1|Item), data = vocabDat, family = 'binomial')

and its equation would look like:

LOij = (beta0 + S0i + I0i) + beta1*TimeDummy + beta2*ViewDummy+ beta3*NoViewDummy + beta4*TimeDummy*ViewDummy+ beta5*TimeDummy*NoViewDummy

where:

  • LOij = log odds of correct response for i-th subject on j-th item;
  • I0i is a random intercept (aka random item effect);
  • all else is as above.

So this last model will also enable you to model the change in odds of a correct response between postest and pretest, separately for each group, after taking into account subject-to-subject and item-to-item variability in responses.

$\endgroup$
11
  • 1
    $\begingroup$ All your assumptions are correct :). Yes,Groups sat Pretest & Posttest before & after exposure and your formula is exactly what we originally started with. My worry is: does such model analyse the response change from pretest to posttest of each participant for each item, then compares the group means of change results? or does it simply compare overall participant score of all items between the two times period, because if it does, then the true gain is not considered(having 5 correct items in both times does not necessarily mean 0 change as we can't tell whether they are the same items). $\endgroup$ – Acer acer Dec 3 '18 at 13:11
  • $\begingroup$ Following my previous comment, If the model takes sum score of all items at Pretest and compare it to that of Posttest, then maybe I should create a new column called Gains with dichotomous data where 1 means 0 at Pretest and 1 at Posttest (Gain) while 0 whenever Posttest is 0 (no gain), and do regression on this new dependent variable? $\endgroup$ – Acer acer Dec 3 '18 at 21:02
  • $\begingroup$ @Aceracer: Can you clarify how your Response is actually defined for each subject by item combination at pretest and postest, respectively? Is it defined as a binary variable at each time occasion (in which case, what would 0 and 1 stand for)? Or is it defined as the number of correct responses for an item (assuming each item involves multiple questions) out of all responses attempted? We need to know more about your Response to answer your questions. From your first comment, it almost looks like you are interested in summing correct responses across items, which doesn't seem right to me. $\endgroup$ – Isabella Ghement Dec 4 '18 at 0:53
  • 1
    $\begingroup$ Please let me know if my question hasn't been clarified enough. I look forward to hear from you on the above :) $\endgroup$ – Acer acer Dec 5 '18 at 11:56
  • 1
    $\begingroup$ I can't thank you enough for your comprehensive explanation :-) $\endgroup$ – Acer acer Dec 5 '18 at 20:00
3
$\begingroup$

1) I think what you are actually after is:

mod1 <- glmer(Response ~ Time*Group + (Time| Group/Subjects/Word)
data= vocabDat, family='binomial')

The "/" implies nesting in the Rogers-Wilkinson formula syntax which will expand out to Group + Group:Subject + Group:Subject:Word - this means that each error is caveat-ed by the higher hierarchical terms and like will be compared to like.

2) I have not used the Rasch models before in the past, although I do deal with biological datasets that have temporal and spatial autocorrelation, which more or less amounts to the same thing. In my case, GLMM models are perfectly suitable for capturing systematic biases but that doesn't mean that one should exclude possibilities. My recommendation would be to try a Rasch model (there is one in R - https://cran.r-project.org/web/packages/eRm/vignettes/eRm.pdf) and see if the two models give you roughly the same results.

Hopefully that helps some what, but if you need me to clarify anything then just comment below.

EDIT: Follow Isabelles answer, not my own! I made a mistake. A more sensible model would be (As Isabella Ghement) suggested would be:

mod1 <- glmer(Response ~ Time*Group +
                     (Time|Subject) +
                     (Time|Item), 
          data = vocabDat, 
          family = 'binomial')

Which allows for crossing, rather than nesting, if that is the correct design.

$\endgroup$
3
  • 2
    $\begingroup$ Andre, can you please have a look at my answer and see if it makes sense to you? It doesn't seem to me that Group is a random grouping factor. If Group is "fixed", it should feature in the fixed effects portion of the model and possubly before the straight bar | in the construct (...|...). If Group is "random", it should feature after the straight bar | in the construct (...|...). The formula you provided has Group simultaneously featured as "fixed" and "random", which is not (?) feasible. Thank you so much! $\endgroup$ – Isabella Ghement Dec 3 '18 at 1:36
  • 2
    $\begingroup$ You are quite right - thank you for pointing out the error; I have amended accordingly! $\endgroup$ – André.B Dec 3 '18 at 3:19
  • 2
    $\begingroup$ That's great, Andre! Many thanks! I think there wasn't enough info in the original answer - we both had to make assumptions to respond and maybe the assumptions we made are justified or maybe the are not. Only the asker of the question knows what is truly going on with their study design - all we can do is to make some educated guesses. 🤗 $\endgroup$ – Isabella Ghement Dec 3 '18 at 3:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.