# Compare treatments on mean difference between two times

This sounds like an easy setup, and I'm absolutely sure it's not too complicated to do, but somehow I can't figure out how to approach this.

The setting is as follows : There are 6 treatments with 16 subjects in each treatment. Weight is measured for every subject before and after treatment.

The hypothesis to be tested : is there a difference between treatments in effect (the difference in weight before and after treatment).

Very simple repeated measures design, but the thing is that the subjects are not identified. The only thing you know is that it's the same subjects, but you can't use a repeated measures due to the lack of some id variable.

Up til now, I constructed the mean difference as the difference between the means at t=0 and t=1 for every tank. Calculating the SE for each of these differences allows me to do a series of T-tests and put a Dunn-Sidak correction on it afterwards. But somehow this is cumbersome and feels not right.

Alternatively, I can substract the mean for each treatment at t=0, and do a classical one-way ANOVA. But I am not sure how I should correct the variances in order to count for the fact that I don't substract a fixed number, but a sample mean.

What is the appropriate model to use here, and if possible, where do I find that function in R?

it is NOT aov(weight~treatment*time), as this will make a comparison between eg t=0/treatment=1 and t=1/treatment=3. That's not what I want.

• Can you distinguish what pre-treatment weight is assigned to each different treatment group? (i.e. I know these 16 observations are pre-treatment weights for treatment A). Also were treatments randomly assigned? – Andy W Oct 7 '10 at 13:03
• The 16 observations are 16 fishes in a tank. The treatment is given to the tank. So the 16 fishes in the tank were weighed before and after treatment. But as the fishes weren't marked, it's impossible to link the weights to a specific individual. – Joris Meys Oct 7 '10 at 13:08
• Your issues with independence of observations go beyond repeated measurements. Your 16 fish within a tank are not independent and the treatment was applied to the tank. You do not have 16 independent pre and post observations. In fact, you have 6 observations assigned to one of 6 treatments. Any analysis that uses the fish as the analytical unit will grossly overestimate your degrees of freedom. At very least, you need to account for the nested structure and the intra-tank correlations. Needless to say, you need to increase your N, which is 6 right now (see pseudo-replication). – Brett Oct 7 '10 at 14:41
• @Brett : I am aware of the fact that my observations aren't independent. But I don't agree with your statement that N=6. That would mean that a clinical study with 4 hospitals involved would have N=4, which obviously doesn't make sense. It's impossible to give fish in the same tank different treatments, as the treatment is in the water. The 16 fish are independent, but the measurements at t=0 and t=1 aren't. I correct for this by calculating the df using Satterthwaite and then divide them by 2, which gives me a df of appx. 14 for the t-tests. – Joris Meys Oct 7 '10 at 15:09
• In your hospital example, it depends on how the patients are randomized. If the hospitals are assigned to the treatments, then your N is the number of hospitals. If patients are randomly assigned within hospitals, then your N is the number of patients. Again, I refer you to the literature on pseudoreplication which you'll find easily. Once you say "the treatment is given to the tank" you are done. Level of treatment implies appropriate level of analysis. Here is the original article masterenbiodiversidad.org/docs/asig3/… There's plenty more. – Brett Oct 7 '10 at 15:31

The one-way ANOVA approach you mention sounds fine to me. Sure the individual change scores aren't going to be the "true change" by any means, but they are better than nothing. If anything the resulting variance in the model should be over estimated as a consequence of this procedure.

In R the easiest way to do ANOVA in simple designs (IMO) is to use ezANOVA (package ez). E.g. ezANOVA(data,dv=.(WeightChange),sid=.(PseudoSubjID),between=.(Time))

I can't quite say anything about instantiation, but another approach might be to find the optimal set of paired scores such that the difference between scores is minimized and then treat that pairing as if it is the true pairing. I want to say that approach should be conservative, minimizing the differences between t0 and t1, but your mileage may vary.

First since you do not know which subjects are which, I think it makes more sense to treat each group of observations (before treatment and after treatment) as seperate observations (i.e. as if you randomly assigned treatment to 96 out of 192 subjects). I know this seems obvious but for heuristic purposes I think it helps clarify the question at hand. While this isn't optimal (regression to the mean) I would say you are still better off than many observational studies (assuming no self-selection into treatment).

My initial thought was if treatments are randomly assigned you might as well treat all the pre-treatments as one big control group. You could then use an OLS framework with dummy variables to estimate the treatment effects (with pre-treatments as the reference category). If you know what pre-treatments go to each treatment you could run an anova to see if any non-neglible differences exist between the mean weight in pre-treatment groups. If there are differences in the pre-treatment groups that are not ignorable you should be able to use a multi level framework.

Since fish are in the same tank a multi-level framework may be appropriate anyway.

• I've been thinking about that as well, but the initial mean weight differs significantly between tanks, and using all t=0 observations as one control makes for a very unbalanced design. On top of that, the hypothesis is formulated in difference between treatments, which cannot be formally tested using a control as reference group. The multi-level framework doesn't help me either for the same reason. – Joris Meys Oct 7 '10 at 13:55
• I don't see why the unbalanced design is a problem, but the mean weight difference is obviously a problem with my suggestion. I would still think the multi-level framework would allow you to assess differences between treatments somehow (I do not know how offhand though). The only other thing I can think of would be to simply graph each control/comparison like this response did (minus the connecting lines) stats.stackexchange.com/questions/2067/… , although I imagine this is not entirely satisfactory. – Andy W Oct 7 '10 at 14:10