I am trying to test whether two populations have different means. Let's call the populations "Glaciated" and "Unglaciated." Each population comprises data collected at a number of rivers (9 for Glaciated and 6 for Unglaciated). Multiple samples were taken from each river over time, with n ranging from 13 to over 500, and the variation between rivers within each population is greater than the variation between the two populations See Example Boxplot.

If I do a straight Welches t-test on Glaciated vs. Unglaciated, the high number of samples of some rivers makes those rivers overrepresented in the mean; I get very high levels of significance that change directionality if I add or remove rivers from either population. If I do a Welches t-test on the two sets of river means, I lose the power granted by replication, and the variance within each river fails to be represented. This test fails to reject the null hypothesis of no difference.

What is the appropriate way to hypothesis test this dataset? I don't want any individual rivers to be overrepresented, but I also don't want to lose the information granted by the high levels of replication of both rivers and sampling instances. Should I be resampling/bootstrapping to make the sample size identical for each river?

Edit1: In response to Dave's comment re: measurements taken at different times: These data come from a US Geological Survey database, and they are the result of USGS taking samples from each river every few months for as long as they have been interested in these rivers. Some sample sets date back to the fifties, and some are under twenty years old. Yes, sampling times are a potential confounding variable. Most of the rivers depict regular seasonal fluctuation, but very little fluctuation on scales of years or more. Rivers being sampled at different years doesn't seem to be a problem, because the full range for each river is represented well by any few years of sampling. Below is a typical pattern:Example Boxplot 2

  • 1
    $\begingroup$ I feel like this could be a textbook example of a random effect. The river is a random effect, and then the group (glaciated vs not) is a fixed effect. I do have concerns about the fact that you’re taking measurements at different points in time, however, and it would help immensely if you could expand on that. $\endgroup$
    – Dave
    Commented Mar 16, 2020 at 21:33


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.