I want to examine growth for 38 participants. I have 2 scores at each of 3 separate phases of instruction - baseline, during instruction, and post instruction. Restated, I have 2 baseline scores, 2 instruction scores, and 2 post scores for each participant. All participants are in one group.

  • How should I test growth?
  • A one-way repeated measures ANOVA, or a two-way repeated measures ANOVA?
  • Should I just put this in as 6 consecutive scores (e.g. Time 1-6) and look at the factor of time, or should I look at an instruction phase by time interaction. Does that even make sense?
  • 1
    $\begingroup$ for the 2 scores per phase, are they taken at different time or just repeated measurement (i.e. the second one is taken immediately after the first one)? $\endgroup$ – qoheleth Aug 19 '14 at 0:42
  • $\begingroup$ they are taken at a different time @qoheleth $\endgroup$ – micheal Aug 19 '14 at 4:51

I'd suggest you use Time as a 6-level factor, and then use appropriate contrasts to compare the phases, e.g., Post vs Instruction would be examined using contrast coefficients $(0,0,-.5,-.5,+.5,+.5)$.

It's potentially important to use Time itself in the model because of the repeated measures. For example, some people might want to put some kind of time-series error structure on these -- or possibly a model that assumes greater correlation within a phase than between them.


Toy dataset for 3 subjects, 6 times:

> fake = expand.grid(time=1:6, subj=letters[1:3])
> fake$y = c(18,15,30,28,48,49,19,18,27,28,49,52,19,18,27,25,48,49)

Fit a model with time as a 6-level factor and subj as a random effect:

> library(lme4)
> fake.lmer = lmer(y ~ factor(time) + (1|subj), data = fake)

I'll do it in three stages using the functions in the lsmeans package. First, get the LS~means (aka predictions) for each time

> library(lsmeans)
> (time.lsm = lsmeans(fake.lmer, "time"))

 time   lsmean        SE df lower.CL upper.CL
    1 18.66667 0.8388705 12 16.83890 20.49444
    2 17.00000 0.8388705 12 15.17223 18.82777
    3 28.00000 0.8388705 12 26.17223 29.82777
    4 27.00000 0.8388705 12 25.17223 28.82777
    5 48.33333 0.8388705 12 46.50556 50.16110
    6 50.00000 0.8388705 12 48.17223 51.82777

As a convenience, here are the predictions averaged together in each phase. The function is contrast, but it can estimate linear functions whether or not they are contrasts:

> (phase.lsm = contrast(time.lsm, list(base = c(.5,.5,0,0,0,0), 
+    instr = c(0,0,.5,.5,0,0), post = c(0,0,0,0,.5,.5))))

 contrast estimate        SE   df t.ratio p.value
 base     17.83333 0.5947299 9.86  29.986  <.0001
 instr    27.50000 0.5947299 9.86  46.239  <.0001
 post     49.16667 0.5947299 9.86  82.671  <.0001

Now obtain pairwise comparisons of these:

> pairs(phase.lsm)

 contrast       estimate      SE df t.ratio p.value
 base - instr  -9.666667 0.83666 10 -11.554  <.0001
 base - post  -31.333333 0.83666 10 -37.450  <.0001
 instr - post -21.666667 0.83666 10 -25.897  <.0001

P value adjustment: tukey method for a family of 3 means

Note that I could have gone directly to the types of contrasts I mentioned before. For example:

> contrast(time.lsm, list(`base-instr` = c(.5,.5, -.5,-.5, 0,0)))

 contrast    estimate      SE df t.ratio p.value
 base.instr -9.666667 0.83666 10 -11.554  <.0001
| cite | improve this answer | |
  • $\begingroup$ Can you expand on the contrast coefficients? I use R to do my statistics. Would I just add a column to my data and assign a dummy code for each score depending on what instructional phase the score came from? Then do a two-way anova with time and phase and look at whether the factor of phase is significant? $\endgroup$ – micheal Aug 19 '14 at 4:59
  • $\begingroup$ I edited my answer, adding a toy example. Hope it helps. It is too simple, in that my model is really better suited for a randomized block design rather than a repeated-measures situation. $\endgroup$ – Russ Lenth Aug 19 '14 at 17:39

There seems to be many possible ways to do this, depending on what you want exactly.

The idea of ANOVA or repeated measure ANOVA only makes sense (to me) if you have different treatment groups (say, half of the 38 received different instructions etc.). Since all participants belong to 1 group, it seems to me all you need is a good old paired t(z)-test.

But firstly, you need to define growth. E.g. if you define growth to be the difference between the last measurement and the first, then you can run: t.test(Y6,Y1,paired=TRUE) (where Y6 and Y1 are the measurements at the corresponding time).

If you define growth to the the difference between the last 2 and the first 2, then you can first derive that variable, and reduce the problem to the previous case. Ypost=(Y6+Y5)/2;Ybase=(Y1+Y2)/2 and then t.test(Ypost,Ybase,paired=TRUE)

This is of course the simplest way to do the analyses, there is arguably more sophisticated ways to do things, like a linear mixed model with random participant effect and temporally correlated error structure. But without knowing what exactly you want to do, it seems best to stick with the simpler way (i.e. t.test).

| cite | improve this answer | |
  • $\begingroup$ A simple analysis like the paired t.test, with growth as the difference between the last 2 and the first 2 seems like the best path. This is sort of a preliminary analysis to satisfy an assumption of a larger analysis. I don't think reviewers will be particularly concerned about how I tested for growth given that it is not the focus of the research. Thanks! $\endgroup$ – micheal Aug 19 '14 at 16:04
  • $\begingroup$ Compared with an approach where a model is fitted to all the data, this approach's advantage is that it is somewhat more robust to underlying assumptions (e.g. variance homogeneity); and its disadvantage is that the estimate of variance is cruder (with an overarching model, you get information about variance from all the data, not just the parts used in the paired t test). $\endgroup$ – Russ Lenth Aug 20 '14 at 0:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.