Ecological study with pseudoreplication, should I use t.tests with bonferonni correction or a repeated measures ANOVA?

Question

I have a study where I will treat two ponds and measure changes in soil texture. I only have access to the two ponds, but will collect soil at 15 points in each pond. One pond will be flooded and dried over the course of a year, the other pond will serve as a control and remain flooded throughout. I will sample 6 times over the course of the year.

I would like to compare, at each sampling event whether a difference in soil texture exists. I was intending to do ttests to compare the control to the treated pond at each sampling event, with a bonferonni correction. Does this seem like a good course of action?

POND A Dried and reflooded monthly 15 points are sampled 6 times in the year

POND B Never dried (Control) 15 points are sampled 6 times in the year

Answer 1

You can figure this out by thinking through the baseline assumptions of both your experiment and a Bonferoni Correction.

In this case you have just one comparison, Pond A to Pond B. You are taking 15 samples at each location and using those for your test. You are sampling six times, and each time taking 15 & 15 and performing a test.

This test is separate from the prior tests or subsequent tests, and you are still only testing to see if the modification alters the outcome in Pond B against the control Pond A for this test. So, two subjects.

Bonferoni Correction is used to compensate for the inflated probability of dismissing a null (creating type 1 error) when it is in fact not warranted, because of MULTIPLE comparisons, meaning Pond A to Pond B, Pond A to Pond C, Pond B to Pond C.

As you have described your process this is not the case, so no correction is needed. You can use 6 separate T-Tests and 6 points in time.

As for the repeated measures ANOVA, you would use that on Pond B against Pond B if you tested prior to doing your experimental treatment and then again after treatment on the same subject with more than one repetition.

As you are describing comparing two ponds, not the same pond to itself repeatedly, the repeated measure ANOVA is not warranted.

That having been said, were I doing this experiment, I would want a baseline t-test for Pond A and Pond B, to be sure they did not vary more than one might expect from randomness alone PRIOR to the treatment.

And I might use the repeated measure ANOVA on Pond B after the whole series to gauge changes intrinsic to that system across your pre-sample and 6 sample events. (If I did that I would do it with the Control as well to be sure that there is no natural evolution in texture outside of the experiment that is statistically meaningful)

Answer 2

First, a simple Bonferroni correction is seldom a good idea. It's easy to think about and to code, but it's not as powerful as the Holm modification.

Second, whenever you are doing multiple comparisons you need to account for them in some way. Even if you think about the 15 samples per pond as technical replicates, if you do 6 separate comparisons between the ponds you need some correction for those 6 comparisons.

The most powerful way to do this would be to go beyond either of the methods you propose. Repeated-measures ANOVA will pose a problem if you ever miss a measurement. Consider generalized least squares, or a mixed model that treats sites within a pond as random effects. Continuous modeling with respect to time, rather than 6 separate comparisons between the ponds, would minimize the multiple comparisons problem.