# Repeated measures over time with small $n$

I was given data to analyze for a study looking at the effects of a treatment on iron levels at four different time points (before treatment, the day treatment ended, 4 weeks after treatment, and 2-4 months after treatment). There is no control group. They are looking to see if there are significant increases in iron levels at each of the 3 post-treatment time points to the before treatment (baseline) level. Eleven patients had baseline levels but only 8 patients had complete data for all 4 time points ($n$=11, 10, 9, and 8 for each time point). Not only were iron levels measured, but two other laboratory measures were taken at each time point to be compared to baseline.

I have a few questions as to how to analyze this. I first thought an RM ANOVA would be appropriate to analyze this data, but I was concerned about the small sample size, the loss of data, and the non-normal distribution of the data. I then considered comparing each post-treatment measure to baseline using Wilcoxon signed-rank tests, but then I run into the issue of multiple comparisons. However, I have read some literature that downplays needing to run multiple comparisons. So overall, I'm dealing with small sample sizes, incomplete data, and multiple comparisons (and whether or not its necessary).

I hope this all made sense. I'm new to CrossValidated and was directed here by a colleague as a place to learn from experienced statisticians, so I'd appreciate any advice! Thanks!

Edited to add raw data from comment:

There are four total time points and the outcome variable is continuous. For example, the results at each time point look similar to this:

 Baseline (n=11): [2, 7, 7, 3, 6, 3, 2, 4, 4, 3, 14]
1st Post (n=10): [167, 200, 45, 132, ., 245, 199, 177, 134, 298, 111]
2nd Post (n=9):  [75, 43, 23, 98, 87, ., 300, ., 118, 202, 156]
3rd Post (n=8):  [23, 34, 98, 112, ., 200, ., 156, 54, 18, .]

• If you add reproducible example (or raw data) it would be helpfull. – Ladislav Naďo Oct 3 '13 at 11:44
• The outcome variable is continuous. For example, the results at each time point look similar to this: Baseline levels n=11: [2, 7, 7, 3, 6, 3, 2, 4, 4, 3, 14]. 1st Post n=10 [167, 200, 45, 132, ., 245, 199, 177, 134, 298, 111]. 2nd Post n=9 [75, 43, 23, 98, 87, ., 300, ., 118, 202, 156]. 3rd Post Level n=8 [23, 34, 98, 112, ., 200, ., 156, 54, 18, .] . There are four total time points. – msturm17 Oct 3 '13 at 14:02

I have re-think your problem and found Friedman's test which is a non-parametric version of a one way ANOVA with repeated measures.

I hope you have some basic skills with R.

# Creating a source data.frame
my.data<-data.frame(value=c(2,7,7,3,6,3,2,4,4,3,14,167,200,45,132,NA,
245,199,177,134,298,111,75,43,23,98,87,NA,300,NA,118,202,156,23,34,98,
112,NA,200,NA,156,54,18,NA),
post.no=rep(c("baseline","post1","post2","post3"), each=11),
ID=rep(c(1:11), times=4))

# you must install this library
library(pgirmess)


Perform test Friedman's test...

friedman.test(my.data$value,my.data$post.no,my.data$ID) Friedman rank sum test data: my.data$value, my.data$post.no and my.data$ID
Friedman chi-squared = 14.6, df = 3, p-value = 0.002192


and then find between which groups the difference exist by non-parametric post-hoc test. Here you have all possible comparisons.

friedmanmc(my.data$value,my.data$post.no,my.data$ID) Multiple comparisons between groups after Friedman test p.value: 0.05 Comparisons obs.dif critical.dif difference baseline-post1 25 15.97544 TRUE baseline-post2 21 15.97544 TRUE baseline-post3 20 15.97544 TRUE post1-post2 4 15.97544 FALSE post1-post3 5 15.97544 FALSE post2-post3 1 15.97544 FALSE  As you can see only baseline (first time point) is statistically different from others. I hope this will help you. • Ladislav, this an excellent answer to this question. It is extremely thorough and complete. The only issue I see is that Kruskal-Wallis ANOVA's also have an assumption of independence of observations, such that there are different subjects at each level of the independent variable, which in this case, we do not have as we are following the same 11 patients across 4 time points. Do you have any opinion on this or have any other methods in mind to address this issue? Thanks so much! – Matt Reichenbach Oct 3 '13 at 17:04 • I deleted my comment above. I finally found better test. Enjoy ! – Ladislav Naďo Oct 9 '13 at 14:57 • This isn't my original question, @msturm17, will have to accept your answer, I gave you the bounty though! – Matt Reichenbach Oct 9 '13 at 16:56 • Thank you, Ladislav, for taking the time to thoroughly respond to my question! – msturm17 Oct 11 '13 at 12:15 If you don't know the distribution of individual changes over time, you cannot approximate it with distribution of between-patient differences. For example, if you have 10 patient with respective iron levels (510,520,...,600) before treatment and (520,530,...,610) after treatment, the Kruskal-Wallis ANOVA (or any other similar algorithm) would claim that there is no significant change of iron levels. IMHO, without the control group, the best you can do is to count how many patients increased their iron level and how many decreased it, and test the significance of this. That said, if the KW ANOVA tells you that there is a significant iron level, it is (no false positives). • Yay for no false positives! Haha, thanks for your answer. How do you suggest we "test the significance of this" in regards to counting how many patients increased their iron level and how many decreased it? Thanks! – Matt Reichenbach Oct 9 '13 at 13:39 • If there are$m$positives and$n$negatives,$p = 2^{-(m+n)} \sum_{k=0}^m {m+n \choose k}\$ – user31264 Oct 9 '13 at 14:25
• Thank you! This was another interesting way to look at my question and will see how it applies to my data. – msturm17 Oct 11 '13 at 12:39