# Combining repeated experiments into one dataset

I hope you all won't mind a basic question.

We are examining the effects of a compound at various concentration on the behaviour of an organism. The compound is administered once at the beginning of the time course. Observations are made every minute for a period of 2 hours. Each concentration is applied to 12 individuals.

If we repeat this experiment with different individuals, what is the best way to combine the 2 experiments? The issue that we're having is that although the distribution and shape of the data from each experiment is very similar, the plot of the medians by time is usually significantly shifted up or down between exp1 and exp2.

It will no doubt be very evident that I'm unskilled in statistics. If this question needs more information to be answerable, let me know.

Thanks, Jen

Just add "experiment" as an effect to your model, that should account for the shift between experiments and let you gain the power of increased N across experiments to detect effects of concentration and time.

In R, if using ANOVA and treating time as a factor (e.g. not numeric), then do:

library(ez)

ezANOVA(
data = my_data
, dv = .(my_dv)
, wid = .(individual)
, within = .(time)
, between = .(concentration,experiment)
)


However, this:

1. treats experiment as a fixed effect, whereas it might more reasonably be considered a random effect (thanks Henrik!)
2. treats time as non-continuous
3. assumes sphericity across the levels of time

An approach that solves all three issues is to employ a mixed effects model. If you think that the effect of time is linear, then leave time as a numeric variable and do:

library(lme4)

lmer(
data = my_data
, formula = my_dv ~ time*concentration+(1|individual)+(1|experiment)
)


If you don't think time is linear, you could convert it to a factor and repeat the above, or use generalized additive mixed modelling:

library(gamm4)
fit <- gamm4(
data = my_data
, formula = my_dv ~ time+concentration+s(time,by=concentration,bs='tp')
, random = ~ (1|individual) + (1|experiment)
)
print(fit\$gam)


That assumes that experiment only shifts the time function, but lets concentration change the shape of the time function. I have a hard time figuring out how to visualise the results from single gamm4 fits, so I usually obtain the fitted model's predictions across the fixed-effects space then bootstrap (in your case, sampling individuals with replacement within each experiment) confidence intervals around these predictions.

Also, all of the above assume that residuals are gaussian; if you're dealing with anything different (eg. binomial data), then you need to change the "family" arguments of lmer and gamm4 (ezANOVA can't do anything but gaussian).

• I have never heard of this approach, but it seems somehow reasonable. But, "experiment" should rather be a random factor, than a fixed one. These two experiments are just two randomly drawn from the population of possible experiments with this design. They are in no way fixed. Sep 17, 2010 at 13:32
• Ah, you're indeed correct that experiment should be a random factor. I'll update the answer to reflect this. Thanks! Sep 17, 2010 at 13:35

I think a way in which you could analyze the two experiments together is by defining a multilevel/hierarchical model. The individuals are nested within each experiment.

The standard for this approach is Gelman's book (I think).
The Journal of Memory and Language had a special issue on analyzing data in 2008 which covered hierarchical models and even some examples in R.
Other's here can probably provide good web-resources as well.

• Baayen's textbook, Practical Data Analysis for the Language Sciences with R, may still be found at bit.ly/aQ1KGb
– chl
Sep 17, 2010 at 13:59
• Thanks. I just had it in a kind of unusable Version with "Draft" in the background. Sep 17, 2010 at 17:34