# Statistical test to compare multiple treatments on the same 'individual' over time

I'm looking for a statistical test or tests to compare the effects of different 'treatments' on a single 'individual' over time. The single 'individual' could be one person or a group acting as a single entity (e.g. a team, a company, etc.). A simple example is comparing the effects of two different exercise regimes on one person's weight loss.

The main concern I have with using a 'standard' statistical test (in this case, something like a t-test, Mann-Whitney U test or Kolmogorov-Smirnov Test seems appropriate) is the obvious non-independence of the data.

I could check for independence by running some form of ABAB study (i.e. alternating the regimes several times to check that they are not interacting). This seems to be the approach that the author of this 'Self Experimentation' paper (Roberts, S.) seems to take. He then uses repeated chi-squared tests (first A against first B, second A against first B, etc.). Is it a valid statistical approach to perform multiple tests on the same data or does this increase the chance of type I errors as indicated in this paper?

The more often one analyses the accumulating data, the greater the chance of eventually and wrongly detecting a difference, thereby drawing incorrect conclusions from the trial.

Should I worry about the non-independence? Can I simply perform multiple tests against the same data? Is there a specialised test for this kind of experiment?

• It would help to know more about your specific situation and the specific question you want to answer. If you do an ABBA study (perhaps repeatedly) you might be able to calculate the difference between the first A and B, the difference between the second A and B, and so on. You could then do a t test to see if these differ from zero. Aug 23, 2012 at 12:35

If the time aspect is very important then you may want to consider some type of structured dependency model that can deal with a time series, but if I'm understanding your question correctly here's my answer.

The problem with conducting those standard statistical tests repeatedly is called alpha inflation or the problem of multiple comparisons.

Analysis of Variance is a procedure that allows conducting those standard tests simultaneously. The most basic version is a generalized t-test. There are variations of ANOVA like Kruskal-Wallis which is nonparametric. ANOVA is discussed thoroughly on this site. Here is an example of how to do it in SAS, SPSS, and R.

Since you're measuring the same individual repeatedly you'll want to use a repeated measures design. If you're testing multiple hypotheses you'll want to also use a correction factor like Bonferroni, but as far as your question seems to be concerned this may not be the case so this last part may be unnecessary. rANOVA can be sensitive to assumption violations though so if you see your data violates them you may need to consider some other linear model.