I was given data to analyze for a study looking at the effects of a treatment on iron levels at four different time points (before treatment, the day treatment ended, 4 weeks after treatment, and 2-4 months after treatment). There is no control group. They are looking to see if there are significant increases in iron levels at each of the 3 post-treatment time points to the before treatment (baseline) level. Eleven patients had baseline levels but only 8 patients had complete data for all 4 time points ($n$=11, 10, 9, and 8 for each time point). Not only were iron levels measured, but two other laboratory measures were taken at each time point to be compared to baseline.

I have a few questions as to how to analyze this. I first thought an RM ANOVA would be appropriate to analyze this data, but I was concerned about the small sample size, the loss of data, and the non-normal distribution of the data. I then considered comparing each post-treatment measure to baseline using Wilcoxon signed-rank tests, but then I run into the issue of multiple comparisons. However, I have read some literature that downplays needing to run multiple comparisons. So overall, I'm dealing with small sample sizes, incomplete data, and multiple comparisons (and whether or not its necessary).

I hope this all made sense. I'm new to CrossValidated and was directed here by a colleague as a place to learn from experienced statisticians, so I'd appreciate any advice! Thanks!

Edited to add raw data from comment:

There are four total time points and the outcome variable is continuous. For example, the results at each time point look similar to this:

 Baseline (n=11): [2, 7, 7, 3, 6, 3, 2, 4, 4, 3, 14] 
 1st Post (n=10): [167, 200, 45, 132, ., 245, 199, 177, 134, 298, 111]
 2nd Post (n=9):  [75, 43, 23, 98, 87, ., 300, ., 118, 202, 156]
 3rd Post (n=8):  [23, 34, 98, 112, ., 200, ., 156, 54, 18, .]
  • 1
    $\begingroup$ If you add reproducible example (or raw data) it would be helpfull. $\endgroup$ Oct 3, 2013 at 11:44
  • $\begingroup$ The outcome variable is continuous. For example, the results at each time point look similar to this: Baseline levels n=11: [2, 7, 7, 3, 6, 3, 2, 4, 4, 3, 14]. 1st Post n=10 [167, 200, 45, 132, ., 245, 199, 177, 134, 298, 111]. 2nd Post n=9 [75, 43, 23, 98, 87, ., 300, ., 118, 202, 156]. 3rd Post Level n=8 [23, 34, 98, 112, ., 200, ., 156, 54, 18, .] . There are four total time points. $\endgroup$
    – msturm17
    Oct 3, 2013 at 14:02

2 Answers 2


I have re-think your problem and found Friedman's test which is a non-parametric version of a one way ANOVA with repeated measures.

I hope you have some basic skills with R.

# Creating a source data.frame
post.no=rep(c("baseline","post1","post2","post3"), each=11),
ID=rep(c(1:11), times=4))

# you must install this library

Perform test Friedman's test...


    Friedman rank sum test

data:  my.data$value, my.data$post.no and my.data$ID
Friedman chi-squared = 14.6, df = 3, p-value = 0.002192

and then find between which groups the difference exist by non-parametric post-hoc test. Here you have all possible comparisons.

Multiple comparisons between groups after Friedman test 
p.value: 0.05 
               obs.dif critical.dif difference
baseline-post1      25     15.97544       TRUE
baseline-post2      21     15.97544       TRUE
baseline-post3      20     15.97544       TRUE
post1-post2          4     15.97544      FALSE
post1-post3          5     15.97544      FALSE
post2-post3          1     15.97544      FALSE

As you can see only baseline (first time point) is statistically different from others.

I hope this will help you.

  • 1
    $\begingroup$ Ladislav, this an excellent answer to this question. It is extremely thorough and complete. The only issue I see is that Kruskal-Wallis ANOVA's also have an assumption of independence of observations, such that there are different subjects at each level of the independent variable, which in this case, we do not have as we are following the same 11 patients across 4 time points. Do you have any opinion on this or have any other methods in mind to address this issue? Thanks so much! $\endgroup$ Oct 3, 2013 at 17:04
  • $\begingroup$ I deleted my comment above. I finally found better test. Enjoy ! $\endgroup$ Oct 9, 2013 at 14:57
  • $\begingroup$ This isn't my original question, @msturm17, will have to accept your answer, I gave you the bounty though! $\endgroup$ Oct 9, 2013 at 16:56
  • $\begingroup$ Thank you, Ladislav, for taking the time to thoroughly respond to my question! $\endgroup$
    – msturm17
    Oct 11, 2013 at 12:15

If you don't know the distribution of individual changes over time, you cannot approximate it with distribution of between-patient differences. For example, if you have 10 patient with respective iron levels (510,520,...,600) before treatment and (520,530,...,610) after treatment, the Kruskal-Wallis ANOVA (or any other similar algorithm) would claim that there is no significant change of iron levels.

IMHO, without the control group, the best you can do is to count how many patients increased their iron level and how many decreased it, and test the significance of this.

That said, if the KW ANOVA tells you that there is a significant iron level, it is (no false positives).

  • 1
    $\begingroup$ Yay for no false positives! Haha, thanks for your answer. How do you suggest we "test the significance of this" in regards to counting how many patients increased their iron level and how many decreased it? Thanks! $\endgroup$ Oct 9, 2013 at 13:39
  • 2
    $\begingroup$ If there are $m$ positives and $n$ negatives, $p = 2^{-(m+n)} \sum_{k=0}^m {m+n \choose k}$ $\endgroup$
    – user31264
    Oct 9, 2013 at 14:25
  • $\begingroup$ Thank you! This was another interesting way to look at my question and will see how it applies to my data. $\endgroup$
    – msturm17
    Oct 11, 2013 at 12:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.