# Paired test, unknown sample identites

I have data in which I have repeated measures on samples (at different times), but do not know the identities of the individual samples at each time point.

Just for some context, I'm doing behavioral work in zebrafish. The fish are in tanks of ~10, however, I do not currently have a reliable way to individually identify the fish. I obtain baseline behavioral data for each fish in a tank and then compare this to their behavior after a specified manipulation. So, the data are amenable to a repeated measures design, if I only knew the identities at each time point.

I'm thinking one way to handle this is to do permutation/randomization tests where I compare the difference in behavior before and after the manipulation to the distribution from the permutation test. However, would I be violating the assumption of the data being independently and identically distributed?

Any thoughts or suggestions? Would really like to find a test that could leverage the fact that the measurements at each time point are not strictly independent to get some of the increased power of repeated measures/paired tests.

• Would it be possible to make a comparison between tanks? That way you can know which observations are independent of one another (although you lose power by modeling groups rather than individuals) – Jautis Jan 28 '16 at 18:10
• Sort of. The challenge is that there tends to be a fair amount of tank to tank variability in the behavioral measure for whatever reason (i.e, even during baseline I see differences between tanks at times). So I'm looking for a way to appropriately 'normalize' the data within a tank, by, for example, looking at before and after data within a tank before comparing between tanks. – jkenney9 Jan 28 '16 at 18:46

I think a repeated design ANOVA would be suitable here. That is, as long as the assumption of sphericity is met in your data.

In the repeated design ANOVA, you can still benefit from your knowledge of tank groups, by using the tank id to capture tank related variability within those groups.

Here is an example in R. I generated data simulating 3 different effect types.

1) difference over time points,

2) interaction between time points and tank - would identify if something in the tanks is mediating the effects of the dependent variable.

3) difference due only to tanks - say for example if you unknowingly had wildly different tank water temperatures.

# example, with 5 tanks, 4 time points, and 5 fish in each tank:
#
# Data frame is set up like this:
#
#       tank_id time_point dep_var
#   1         1          1    0.25
#   2         1          1    0.05
#   3         1          1    0.25
#   4         1          1    0.15
#   5         1          1    0.45
#   6         1          2    0.30
#   7         1          2    0.60
#   8         1          2    0.50
#   9         1          2    1.00
#   10        1          2    0.90
#   11        1          3    0.90
#   ...
#   91        5          3    3.00
#   92        5          3    2.25
#   93        5          3    2.25
#   94        5          3    7.50
#   95        5          3    1.50
#   96        5          4    4.00
#   97        5          4    9.00
#   98        5          4    9.00
#   99        5          4    1.00
#   100       5          4    7.00

tank_ids = sort(rep(1:5, 20))
time_points = rep(sort(rep(1:4, 5)),5)

# three examples:

# 1) with variance sourced from time point
# generate dataset with variance sourced from time point
measurement = sample.int(100, n=10, replace = TRUE) * (time_points/max(time_points))
# build data frame for this test
myData <- data.frame(tank_id = tank_ids,
time_point = time_points,
dep_var = measurement)
summary(aov(dep_var~tank_id*time_point+Error(tank_id),myData))
# Df Sum Sq Mean Sq F value  Pr(>F)
# time_point          1  256.7  256.69   72.20 2.5e-13 ***
# tank_id:time_point  1    2.5    2.53    0.71   0.401
# Residuals          96  341.3    3.56

# 2) with variance sourced from interaction
# generate dataset with variance sourced from time point and tank_ids interaction
measurement = sample.int(100, n=10, replace = TRUE) * (tank_ids/max(tank_ids)) * (time_points/max(time_points))
# build data frame for this test
myData <- data.frame(tank_id = tank_ids,
time_point = time_points,
dep_var = measurement)
summary(aov(dep_var~tank_id*time_point+Error(tank_id),myData))
# Df Sum Sq Mean Sq F value   Pr(>F)
# time_point          1  82.66   82.66  48.610 3.96e-10 ***
# tank_id:time_point  1  13.78   13.78   8.105   0.0054 **
# Residuals          96 163.25    1.70

# 3) with variance sourced from tank_ids only
# generate dataset with variance sourced from tank_ids only
measurement = sample.int(100, n=10, replace = TRUE) * (tank_ids/max(tank_ids))
# build data frame for this test
myData <- data.frame(tank_id = tank_ids,
time_point = time_points,
dep_var = measurement)
# there should be no significant terms on this version, since all variability
# should be partitioned by the tank id error term
summary(aov(dep_var~tank_id*time_point+Error(tank_id),myData))
# Df Sum Sq Mean Sq F value Pr(>F)
# time_point          1   0.08   0.077   0.024  0.878
# tank_id:time_point  1   1.54   1.537   0.477  0.492
# Residuals          96 309.34   3.222

• Thanks for this suggestion. However, I've always been under the impression (perhaps incorrectly?) that a repeated measures ANOVA is like an extension of a paired t-test in that it assumes you have the identities of the individual observations over each repeated measurement. Is that incorrect? If so, perhaps I'll explore this option a bit more. – jkenney9 Jan 29 '16 at 15:39
• The repeated measure ANOVA is much more flexible, in that it requires group identities, not individual identities (like a paired t-test). After all, the ANOVA is compartmentalizing variation between groups, and those groups can be as small (i.e. individuals), or large (i.e. country), as the analysis (and variability partitioning) requires. – timle Mar 15 '16 at 22:22