4
$\begingroup$

Our study aims to assess whether there has been an increase in overall happiness among participants who used our interventions—such as dance classes, art classes, meditation, and other resources—available prior to the study's start. Once these tools were accessible, we began conducting quarterly surveys with volunteers to track any improvements in their happiness over time. Our initial plan was to use repeated measures ANOVA, but we've encountered issues with participant retention across time points.

For example, some participants joined only in later surveys; some appeared initially and then skipped others, while others returned sporadically. This inconsistency led me to consider alternative approaches, particularly linear mixed models (LMM), which handle missing data more flexibly. For participants who completed at least two surveys across all four-time points, we found:

  • 21 participants in Time 1,
  • 34 participants in Time 2,
  • 45 participants in Time 3,
  • 47 participants in Time 4,

resulting in 37% missing data.

For those who completed at least two surveys across three time points (Time 2, 3, and 4), we found:

  • 32 participants in Time 2,
  • 43 participants in Time 3,
  • 44 participants in Time 4,

resulting in 22% missing data.

I also reviewed generalized estimating equations (GEE), which offer population-averaged estimates suitable for analyzing general trends across the sample rather than individual trajectories. GEE appears robust with missing data, but I’m unfamiliar with its assumptions on correlation structures, like "exchangeable" or "autoregressive".

I'm unsure of the best approach, as I initially thought LMM might be a simple alternative to repeated measures ANOVA. If you have any recommendations, I would greatly appreciate your guidance.

I used to determine the required sample size for Repeated Measures ANOVA:

  • for the four repeats, the n = 24,
  • for the three repeats, the n = 28.
Improve this question
$\endgroup$
4
  • $\begingroup$ Sorry to comment on a side issue, but about your power calculations: did you change the effect size specification from GPower options to "As in Cohen (1988)"? If you used the default "As in GPower 3.0", you likely got a too small sample size estimate. The default uses an uncommon effect size metric that incorporates the assumed repeated-measures correlation into the formula. $\endgroup$
    – Sointu
    Commented Nov 16 at 10:25
  • $\begingroup$ @Sointu Hi, thank you for the message. Yes, I used the default settings. I was trying to figure out if that option is justifiable, and I have also contacted the GPower creators, who recommended that it be. If you think using the default option is an issue, I'm happy to hear your suggestion. $\endgroup$
    – anna eyre
    Commented Nov 16 at 19:24
  • $\begingroup$ @Sointu Hi, I was reading about the differences between the effect size specifications in GPower. Do you think using the default option is the wrong choice? Or can it be somehow justified? I've seen published papers where the default option was used for sample size calculation. Thank you for your insight. $\endgroup$
    – anna eyre
    Commented Nov 17 at 17:23
  • 1
    $\begingroup$ In my opinion, yes. Or well, wrong is of course relative, but it gives you a very different sample size estimate from all other software I'm aware of. See here: stats.stackexchange.com/questions/535159/… $\endgroup$
    – Sointu
    Commented Nov 19 at 9:08

1 Answer 1

Reset to default
8
$\begingroup$

Is LMM a good alternative for Repeated Measures ANOVA with Missing Data?

Yes, LMM is a great alternative to repeated measures ANOVA. LMM can handle cases where you have an uneven number of outcome measures on each patient (e.g. some have 1, 2, 3, or more outcome measures) or if the patients are measured at unequal time points (e.g. patient 1 was measured at 3 months, 6 months, 9 months, where patient 2 was measured at 6 months, 1 year, 2 years, etc). Repeated Measures ANOVA cannot handle either of these 2 cases.

LMM assumes the outcome is "missing at random" (MAR) whereas GEE makes the stronger assumption that the outcome is "missing completely at random" (MCAR).

What's the difference?

  • MCAR is exactly what it sounds like, the missing values must be completely random which is often not true in practice.
  • In contrast, MAR means the data must be missing at random after adjusting for covariates that impact missingness. Basically, MAR at least gives you a chance to adjust for the variables that caused missing outcomes (i.e. patient dropout or loss of follow-up).

TLDR: LMM is almost always a better framework than repeated measures ANOVA, it's much more flexible. Given outcomes that are missing on some patients (e.g. patient 2 missed their 6-month follow-up), then LMM is in theory better than GEE because you can adjust for variables that cause the missingness and still have valid inferences, whereas GEE does not have this property.

Improve this answer
$\endgroup$
7
  • $\begingroup$ Thank you so much! Do you think that I can implement LMM to my problem? I'm considering using only three repeats from times 2, 3, and 4 as I will have a smaller percentage of missing values - about 20%, instead of all four repeats, which gives me a percentage of missing values of 36%. $\endgroup$
    – anna eyre
    Commented Nov 14 at 19:17
  • $\begingroup$ @annaeyre: yeah, definitely worth trying LMM with your data. I recommend using all available time points rather than discarding the data. You could try both approaches as a sensitivity analysis to see if the results change though. $\endgroup$
    – jarbet
    Commented Nov 14 at 19:57
  • $\begingroup$ Thank you! My only question is how to account for the fact that I have some participants who did 4 surveys, some who did 3, and just 2. In the case of 4 surveys, then I might have to code time 1 as 0, 2 as 1, 3 as 2, and 4 as 3. But what about those with 3 time points? What if they started at study timepoint 2? I would probably still code their first time point as 0 because that is when they entered the study. For a participant who attends Time 1 and Time 4: Time 1 would be coded as 0 Time 4 would be coded as 1. $\endgroup$
    – anna eyre
    Commented Nov 15 at 22:55
  • $\begingroup$ This coding probably helps ensure that my analysis properly handles the different number of observations per participant while still allowing for comparisons across time. But I'm still unsure if this is the way. And also what if I have a participant who attended 1st, 3rd and 4th survey? Should I code the time as 0,1,2? Or should it be 0,2,3 based on the difference between time 1 and 3 and time 3 and 4? Or should I just not to the coding and keep it the way it is? Thank you for any suggestions. $\endgroup$
    – anna eyre
    Commented Nov 15 at 22:56
  • $\begingroup$ @annaeyre: you need to put your dataset in "long format", where each patient has multiple rows, one row per survey. So you will have a patient.id column to identify patients, and a Time column to identify the time at which they filled out the survey (or the time since they started the intervention?). The other columns can give responses to specific survey questions. Does that help? $\endgroup$
    – jarbet
    Commented Nov 19 at 4:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.