# How can I test for significance of a treatment in an unbalanced, repeated-measures experiment using R?

CrossValidated Community,

I must mention that I am a first-time poster (and relatively new to both modelling and R), so please excuse any norms I may violate in my post and politely inform me.

I have been attempting to determine if a treatment variable is significant in R and to find a package that will both 1) model my data appropriately for its structure, and 2) print me some measure of the significance of the treatment (e.g., a p-value)

The experiment investigates whether or not a fish is sensitive to an odor stimulus. There are only 3 subjects (whale sharks are not easy to come by) and 3 treatment groups (pre-odor control, odor, post-odor control). Trials were performed opportunistically on the days they were conducted, meaning that there is not data for all treatment types for all animals on all days.

In one subset of the experiment we recorded different behavioral responses. Originally this was recorded was recording as 0/1 for each response type, with 0 meaning it was not seen and 1 meaning it was seen. I also manipulated this data to give each response type a separate value (1-4), with higher values indicating more "intense" responses, and added a column to the data which sums these values for a final "score". Finally, I created a data frame that averages these scores over "identical" trials (i.e., same day, same animal, same treatment) Data looks like this:

DATE ID Treatment No.Response Open.Mouth Gulp Tail.down
6/14/2011  2         1           1          0    0         0
6/14/2011  2         1           1          0    0         0
6/14/2011  2         2           1          0    0         0
6/14/2011  1         2           1          0    0         0
6/14/2011  2         2           0          1    0         1
6/14/2011  2         3           1          0    0         0


And like this:

DATE ID Treatment No.Response Open.Mouth Gulp Tail.down score
1  2         1           1          0    0         0     1
1  2         1           1          0    0         0     1
1  2         2           1          0    0         0     1
1  1         2           1          0    0         0     1
1  2         2           0          2    0         4     6
1  2         3           1          0    0         0     1


And like this:

DATE ID Treatment No.Response Open.Mouth Gulp Tail.down score
1  1         2         1.0          0    0         0   1.0
1  2         1         1.0          0    0         0   1.0
1  2         2         0.5          1    0         2   3.5
1  2         3         1.0          0    0         0   1.0
2  1         1         1.0          0    0         0   1.0
2  1         2         1.0          0    0         0   1.0


I originally tried utilizing the EzMixed function in the ez package, but the printout is not desirable. My audience needs a more clear-cut indication of the significance of the treatment variable than an AIC comparing a model with/without the treatment variable included.

I looked in to using the lme4 package, but my understanding is that ezMixed is essentially just a wrapper for this package and that I will likely get the same printout (AIC).

Does anyone have any ideas for how I can find what I am looking for using any of the ways my data is formatted above? The ideal function would allow me to test for the significance of the treatment variable as well as any other variables (date, ID, interactions) and save all permutations of these models in a way that I can later compare by AIC.

I would greatly appreciate any help you can give.

This is the original ezmixed model I was running. I used the first data-type, with the exception that date was coded numerically 1-12. I tested each response individually.

mouth_mix = ezMixed(
data = new2response
, dv = .(Open.Mouth)
, random = .(DATE, ID)
, fixed = .(Treatment)
, family = binomial
)
mouth_mix
print(mouth_mix$summary) > print(mouth_mix$summary)
effect error warning     RLnLu     RLnLr DFu DFr   L10LRa   L10LRb
1 Treatment FALSE   FALSE -55.52405 -62.21555   4   3 4.943573 3.517696


Another subset of the experiment utilized video recording data to determine speed changes after intersecting odor (or control) plumes in the water. The data looks like this:

DATE ID Treatment Speed.Before.Plume Speed.After.Plume Speed.difference
6/14/2011  2         1           24.71041          27.85269        3.1422787
6/14/2011  2         2           21.89439          22.04172        0.1473287
6/14/2011  1         2           23.15754          26.64582        3.4882799
6/14/2011  2         2           22.54465          14.15707       -8.3875764
6/14/2011  2         3           25.80078          18.87284       -6.9279353
6/14/2011  2         3           23.50434          17.67082       -5.8335139


Again, I tried using EzMixed but I want a more clear cut indication of significance for my audience. I am having the same problems modelling this unbalanced, repeated-measure data. The only difference is the dependent variable.