I'm a neuroscientist approaching doing some complex statistics and can't find anyone to point me in the right direction of what I need. I'm happy to read around the subject, I just don't know enough to know what to google right now...
Anyway, here's the problem. It involves some complex background so I've come up with an analogy:
Imagine I had a machine that could detect and measure the depth of every fish in a column of water in a sea. I take it out in my boat and detect 1000 fish. So now I have some univariate data. This turns out to be nothing like a normal distribution - there are groups of fish that hang out at the bottom, groups near the surface, and some in the water column. So reducing these 1000 depth measurements to things like a mean or median doesn't really make sense. Rather, we could derive a single empirical probability density function.
For some reason, I come up with a hypothesis that this function is different in tropical and cold seas. For the purpose of this example, I'm treating the type of the sea as a binary variable, not a continuous one. Imagine I don't own a thermometer.
So I measure my PDFs using my fish depth machine in five cold seas around the coast of the UK - let's call them C1, C2, C3, C4, C5.
I then go on holiday to a tropical island and take 5 PDF readings - let's call them T1, T2, T3, T4, T5.
I understand how to compare any single pair of PDFs - I could, for example, use a two-sample Kolmogorov-Smirnov. However, I want to model the repeats (1-5) as a random effect, and ask the question whether the differences between groups C and T exceeds the differences within each group.
I don't want to pool each group. This is for a couple of reasons. The most obvious is that the number of fish in each group is quite variable. So C1 might have 1000, and C2 might have 10,000. Pooling them will obscure the fact that C1 is a complete experiment whose importance should be equally weighted.
So how can I go about this? Is there anything like a known approach for this, I do I have to resort to a pairwise bootstrapping approach?