Suppose I am running an experiment to see if a treatment changes the mean weight of a group of people. Note that I am specifically interested in the mean weight: if half the people get heavier, and half get lighter, that is considered a null result. There of course may be some random drift, so I employ a control group:


For the treatment, people are organized into k groups with m subjects each, and I have measurements only of the mean: k independent measurements.

For the control, people are also organized into k groups with m subjects each, but I have measurements on every individual. So now I have k*m independent measurements of height. However, this still produces only k independent measurements of my outcome, since I'm interested in the change in the mean, not the change in individuals.


How can I compare the difference in the change in the mean height between the control and treatment, while taking advantage of the fact that I have more independent measurements for the control groups?

We know the variance on the control measurement is lower, but I can't figure out how to actually take advantage of that in a test, bootstrapped or otherwise. The only thing I can do is measure two separate confidence intervals and see if they overlap, but that's not a proper way to do science.


I actually have individual data for the treatment groups, but the people embedded in a social network, so they are not independent. I can't use nested/hierarchical/multi-level models because that still assumes independence at the lowest level.))

  • $\begingroup$ Why do you think that hierarchical model assumes "assumes independence at the lowest level"? It's exchangeability - i.e. a kind of independence conditional on them being in the particular group + other covariates. You could also consider other methods common in cluster randomized trials such as using average group outcomes only. $\endgroup$ – Björn Sep 20 '18 at 7:21
  • $\begingroup$ By "independence at the lowest level" I mean, as you say, a kind of independence conditional on them being in the particular group. But that is not actually the case in my treatment groups. One person in a network can directly influence another person in a network, above and beyond any group level effects. $\endgroup$ – samplesize1 Sep 20 '18 at 22:37
  • $\begingroup$ "You could also consider. . . using average group outcomes only." That is what I currently do, but it is not efficient because it doesn't take advantage of the true independence of the control group. $\endgroup$ – samplesize1 Sep 20 '18 at 22:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.