# Statistical significance testing when distributions are paired, but samples are not

I have a situation like the following:

5 distributions A, B, C, D, E of unknown type from which I can take samples to estimate the mean.

I apply an intervention or treatment to all of the distributions.

I take another set of samples from each of these distributions and again estimate the mean.

Effectively, the distributions are paired, but the samples within the distribution aren't necessarily. How can I test for statistical significance of the treatment effect if the samples aren't paired? If they are, is there a different approach beyond combining all of the samples into a single large distribution and performing e.g. a paired t test?

Welcome to CV!

It's better to think in terms of dependency rather than pairing. If you measure experimental units more than once, you can model this dependency using a mixed model with a random effect for the experimental unit.

A simple example using package lme4 would be:

model <- lmer(y ~ treatment + (1 | distribution))


Where distribution is what you call A to E. Note that this model assumes a normally distributed error term and uses a random intercept.

I'm not quite sure what you mean by A, B, C, D, E being distributions, but regardless, they don't have to be experimental units: They could be locations, time points, different plates, or even different people performing the measurements, each with their own consistent deviation.

In case they are locations or time points, it might be that measurements from A correlate strongest to other measurements from A, then to B, then to C, etc. This sort of autocorrelation should be accounted for using an appropriate covariance structure.