Paired test, unknown sample identites

Question

I have data in which I have repeated measures on samples (at different times), but do not know the identities of the individual samples at each time point.

Just for some context, I'm doing behavioral work in zebrafish. The fish are in tanks of ~10, however, I do not currently have a reliable way to individually identify the fish. I obtain baseline behavioral data for each fish in a tank and then compare this to their behavior after a specified manipulation. So, the data are amenable to a repeated measures design, if I only knew the identities at each time point.

I'm thinking one way to handle this is to do permutation/randomization tests where I compare the difference in behavior before and after the manipulation to the distribution from the permutation test. However, would I be violating the assumption of the data being independently and identically distributed?

Any thoughts or suggestions? Would really like to find a test that could leverage the fact that the measurements at each time point are not strictly independent to get some of the increased power of repeated measures/paired tests.

Answer 1

I think a repeated design ANOVA would be suitable here. That is, as long as the assumption of sphericity is met in your data.

In the repeated design ANOVA, you can still benefit from your knowledge of tank groups, by using the tank id to capture tank related variability within those groups.

Here is an example in R. I generated data simulating 3 different effect types.

1) difference over time points,

2) interaction between time points and tank - would identify if something in the tanks is mediating the effects of the dependent variable.

3) difference due only to tanks - say for example if you unknowingly had wildly different tank water temperatures.

# example, with 5 tanks, 4 time points, and 5 fish in each tank:
# 
# Data frame is set up like this:
#
#       tank_id time_point dep_var
#   1         1          1    0.25
#   2         1          1    0.05
#   3         1          1    0.25
#   4         1          1    0.15
#   5         1          1    0.45
#   6         1          2    0.30
#   7         1          2    0.60
#   8         1          2    0.50
#   9         1          2    1.00
#   10        1          2    0.90
#   11        1          3    0.90
#   ...
#   91        5          3    3.00
#   92        5          3    2.25
#   93        5          3    2.25
#   94        5          3    7.50
#   95        5          3    1.50
#   96        5          4    4.00
#   97        5          4    9.00
#   98        5          4    9.00
#   99        5          4    1.00
#   100       5          4    7.00

tank_ids = sort(rep(1:5, 20))
time_points = rep(sort(rep(1:4, 5)),5)

# three examples:

# 1) with variance sourced from time point
  # generate dataset with variance sourced from time point
  measurement = sample.int(100, n=10, replace = TRUE) * (time_points/max(time_points))
  # build data frame for this test
  myData <- data.frame(tank_id = tank_ids,
                       time_point = time_points,
                       dep_var = measurement)
  summary(aov(dep_var~tank_id*time_point+Error(tank_id),myData))
  # Df Sum Sq Mean Sq F value  Pr(>F)    
  # time_point          1  256.7  256.69   72.20 2.5e-13 ***
  # tank_id:time_point  1    2.5    2.53    0.71   0.401    
  # Residuals          96  341.3    3.56           


# 2) with variance sourced from interaction
  # generate dataset with variance sourced from time point and tank_ids interaction
  measurement = sample.int(100, n=10, replace = TRUE) * (tank_ids/max(tank_ids)) * (time_points/max(time_points))
  # build data frame for this test
  myData <- data.frame(tank_id = tank_ids,
                       time_point = time_points,
                       dep_var = measurement)
  summary(aov(dep_var~tank_id*time_point+Error(tank_id),myData))
  # Df Sum Sq Mean Sq F value   Pr(>F)    
  # time_point          1  82.66   82.66  48.610 3.96e-10 ***
  # tank_id:time_point  1  13.78   13.78   8.105   0.0054 ** 
  # Residuals          96 163.25    1.70    

# 3) with variance sourced from tank_ids only
  # generate dataset with variance sourced from tank_ids only
  measurement = sample.int(100, n=10, replace = TRUE) * (tank_ids/max(tank_ids))
  # build data frame for this test
  myData <- data.frame(tank_id = tank_ids,
                       time_point = time_points,
                       dep_var = measurement)
  # there should be no significant terms on this version, since all variability 
  # should be partitioned by the tank id error term
  summary(aov(dep_var~tank_id*time_point+Error(tank_id),myData))
  # Df Sum Sq Mean Sq F value Pr(>F)
  # time_point          1   0.08   0.077   0.024  0.878
  # tank_id:time_point  1   1.54   1.537   0.477  0.492
  # Residuals          96 309.34   3.222