I am new to statistics and have to do some analysing for a beginner project. However, I am so confused right now and have nobody that I can ask.

About the project experiment: We conducted a between-subjects-design with repeated measures. There were three conditions (A, B, C), 9 blocks each. Each participant was assigned to one of these conditions.

My hypothesis: My hypothesis is that the participants will be best in C, worse in B and the worst in A (C>B>A). Also, I hypothesize that the correctness will improve with increasing block-number.

My data: So I have a dataframe:


participant    condition    block   correctness
1              A             1       0.87
2              B             2       0.74

My question now is:

Does it make any sense to do an ANOVA or should I do another analysis?

What test should I use after the ANOVA?
(->three pairwise t-tests in order to compare A-B, B-C, A-C,
-> a Tukey HSD, or
-> other ANOVAs where I do pairwise conditions)?

I already conducted an ANOVA:

summary(aov(correctness ~ condition * block + Error(participant/block), dat))

Error: participant
           Df Sum Sq Mean Sq F value Pr(>F)
condition   2  0.295 0.14761   1.711  0.186
Residuals 100  8.629 0.08629               

Error: participant:block
                 Df Sum Sq Mean Sq F value Pr(>F)    
block             8  4.619  0.5774  44.350 <2e-16 ***
condition:block  16  0.312  0.0195   1.496 0.0942 .  
Residuals       800 10.416  0.0130 

So, I conclude that there is a main effect of block: Block number influences correctness. Also, the correctness does not seem to differ in the conditions.

However, I also conducted pairwise t-tests, and they say that, f.e., Condition A and B differ significantly from each other.

I don't get why the ANOVA says that the conditions do not differ but the t-tests states otherwise? Also, I am really not sure whether I used the right test and which test I should conduct as post-hoc tests?

  • 1
    $\begingroup$ There are tests for a specific ordered alternative; nonparametric tests for them are much better known (Jonckheere-Terpstra probably being the most widely known), but ordered-alternative tests that assume normality do exist if you need them. $\endgroup$ – Glen_b Mar 26 at 23:46
  • $\begingroup$ @Glen_b: See restriktor.org and stats.stackexchange.com/questions/169419/… $\endgroup$ – kjetil b halvorsen Mar 27 at 2:54
  • $\begingroup$ Thank you, do you know by any chance the name of the tests that assume normality? $\endgroup$ – a.henrietty Mar 27 at 10:35
  • $\begingroup$ kjetil's answer at the link he gave seems to cover things quite well. Also see Barlow, Bartholemew, Bremner and Brunk, Statistical inference under order restrictions; there are numerous more recent references as well (e.g. see also work on isotonic regression or monotonic regression). There's also a very simple thing you can do with ordinary ANOVA output (akin to computing a one-tailed p-value by halving the p-values in output from a two-tailed test if the sample means are in the previously predicted order), ... ctd $\endgroup$ – Glen_b Mar 28 at 0:51
  • $\begingroup$ ctd ... but it has an issue under the alternative if some of the ordered population means are (nearly ) equal, since the corresponding sample means might just invert the order; PAVAS / isotonic approaches are better. $\endgroup$ – Glen_b Mar 28 at 0:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.