I completed a pilot trial of 9 participants who all received the intervention and I'm looking at change in behavior over time - using continuous, repeated measures that were given at 3 different time points (pretest, posttest, followup).
I'm wondering, given the small sample size - what's the best statistical test to use?
Thanks for your help!
You have data that are continuous but not necessarily normal. The pre, post and follow-up times can be considered as 'treatments', and the nine participants as 'blocks'.
Friedman test. My short answer is to recommend you use a Friedman nonparametric test. It will test whether pre, post and follow-up scores have any significant differences among them. (One pattern of differences might be that 'post' scores are larger than 'pre' scores, with 'follow-up' scores being a little smaller than 'post" scores.)
Details using R. If you use R, you can look at R documentation for 'friedman.test' to see the syntax for use of its version of the test. However, the Friedman test is used sufficiently widely that many statistical software packages implement it. (There may be slight differences in the implementations, but these likely center on handling tied observations, which should be rare for continuous-scale data.)
No test of inter-subject differences. The Friedman test does not test whether there are differences among subjects. Presumably you have assumed that there are, or you would not have bothered to look at 9 subjects.
Ad hoc tests. If you find significant differences among pre, post, and follow-up, you can use paired Wilcoxon signed-rank tests to investigate the pattern of differences. There are three pairs to look at, so you are working at the 5% significance level, you should not declare such a Wilcoxon test to give a significant result unless the P-value is smaller than 5/3 = 0.017 percent. (This is according to the Bonferroni method of controlling for false discovery.)
Example using R. In case you are not comfortable using R, here is R output for the Friedman test on some fake data, roughly to your specifications.
set.seed(322)
pre = rgamma(9, 5, .1)
pst = pre + runif(9, -1, 6)
fol = pre + runif(9, -1, 3)
pre = round(pre, 3); pst = round(pst, 3); fol = round(fol, 3)
cbind(pre, pst, fol)
pre pst fol
[1,] 42.998 47.810 42.678
[2,] 69.045 73.262 70.906
[3,] 64.608 65.254 67.316
[4,] 12.308 13.891 12.095
[5,] 40.563 46.459 40.539
[6,] 71.083 77.032 73.839
[7,] 51.743 55.181 51.671
[8,] 44.999 45.279 46.151
[9,] 55.848 58.257 54.905
score = c(pre, pst, fol)
time = rep(1:3, each=9)
subj = rep(1:9, 3)
friedman.test(score, time, subj)
Friedman rank sum test
data: score, time and subj
Friedman chi-squared = 8.2222, df = 2, p-value = 0.01639
# significant differences among 'times'
wilcox.test(pre, pst, pair=T)
.
# Ad hoc tests:
Wilcoxon signed rank test
data: pre and pst
V = 0, p-value = 0.003906 # Signif diff (Bonferroni)
alternative hypothesis: true location shift is not equal to 0
wilcox.test(pst, fol, pair=T)
Wilcoxon signed rank test
data: pst and fol
V = 41, p-value = 0.02734 # suggestive. not signif
alternative hypothesis: true location shift is not equal to 0
wilcox.test(pre, fol, pair=T)
Wilcoxon signed rank test
data: pre and fol
V = 15, p-value = 0.4258 # non-significant
alternative hypothesis: true location shift is not equal to 0
