6
$\begingroup$

I'm analyzing a dataset on the Nurse Licensure Exam, comprising 3000 participants. (n)

These 3000 participants were randomly recruited from 13 Sites across the US. (group level variable)

About 40% of the subjects have multiple observations, while the remaining 60% have one observation each. (repeated measure)

My objective is to assess the impact of a certain exposure (A) on participants' test scores, with Site serving as the grouping variable.

Although the (ICC) based on SubjectID suggests moderate correlation (0.5-0.7), I've opted not to include SubjectID as a random component in the model due to lack of interest in predicting individual scores over time. However I am including Site as a random intercept and random slope. (random component)

This is the snapshot of my data below

  ID       date/time         Score(y)          Age(X1)       Race     SiteId
  1        2019-03-20        48                25            Hispanic    TN
  1        2019-10-19        70                25            Hispanic    TN
  2        2020-11-13        65                32            Black       IA
  3        2018-02-18        61                24            White       CO
  .        .                 .                 .             .           .
  .        .                 .                 .             .           .
  3000     2021-05-14        80                23            White       MN

What alternative way can I capture the subject correlation if I am not including this as a random component. Please pardon my ignorance on this topic. Thanks in advance for any advise.

$\endgroup$

2 Answers 2

7
$\begingroup$

If you want to capture the subject correlation in a multilevel model, I think you have to include it as a random effect, and this doesn't really depend on wanting to predict individual scores over time. I'd guess that the majority of people using a random effect of individual are not interested in modeling individual scores, but they need to include this because the scores are correlated.

An alternative is generalized estimating equations (GEE) which is a marginal model, but I don't know much about them. Others here surely do. My link goes to its tag, which you can browse to see if it's useful.

$\endgroup$
1
  • $\begingroup$ That makes sense to me. $\endgroup$ Mar 29 at 14:24
6
$\begingroup$

Although the (ICC) based on SubjectID suggests moderate correlation (0.5-0.7), I've opted not to include SubjectID as a random component in the model due to lack of interest in predicting individual scores over time.

Not a good idea. If you don't include the subject effect, you implicitly assume observations as independent for which you have a clear indication that they are not. The implication of this is that your inference will be based on implicitly assuming a larger effective sample size (namely an amount of independent information of the number of observations) than you actually have (due to within-subject dependence). This means that the variation of your sample estimators will be underestimated. In other words, your results will look more precise than they actually are.

(I'm saying the same thing here as @PeterFlom in different words, hoping that this may help.)

Now in order to make things more complicated, I add that although people routinely include a random effect for this reason, a random effect models a rather specific form of dependence between observations of the same group (subject), namely that dependence comes in the form of an additive constant per group. This cannot be taken for granted in principle. However any modelling of dependence beyond this becomes quite a bit more complicated and particularly often requires a bigger number of observations per group, such as modelling subject-wise growth curves, time series structure, or differing within-subject variances.

The within-group random effect is popular because it is the simplest device to model dependence, and it captures the simplest kind of dependence, which is positive (!) correlation. In many real situations data are such that this kind of dependence can be detected, but nothing more sophisticated, so then modelling a random effect captures whatever dependence can be seen. This does not mean that dependence really plays out like this, but it also means that it is pointless to model more sophisticated dependence because it couldn't be identified from the data.

In fact, assuming a within-group random effect is equivalent to inducing a constant correlation between any two observations within a group. One could even write this down as model without involving a random effect (maybe this is what you have in mind when asking about "capture correlation without including a random effect"), but this wouldn't help you, because it is essentially the same thing and inference would be equivalent.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.