I want to assess the statistical significance between the difference in the mean of two datasets D1 and D2. However, I each data set, the measurements aren't independent: one data set is the collection of measurements done in different different individuals in different sessions. In each session one can have many different values of the same measurement. In fact is more complicated than that, because ideally I would like to detect statistical differences in a variable (lets say blood pressure along the day, for test group and placebo group).

I think that one way to see if the difference of the means of group 1 and 2 is statistically significant is to do some kind of nested bootstrap:

If there are N_I1 individuals named (1, 2, 3,...I1) in group 1 and N_I2 individuals in group 2 ,named (I1+1, I1+2, ...,I1 +I2), and each subject went to S different sessions, I have, for each bootstrap to : -take with replacement N_I1 individuals from the pooled group(I1+I2) -take with replacement N_I2 individuals from the pooled group(I1+I2)
-for each individual, take with replacement S measures from the pooled measures N_I1+ N_I2.

Finally see where does the difference between measures the two groups in my real data real data fits with respect to the distribution of bootstraps.

Does this makes sense ? Thanks for your patience, Im new in this forum.

  • $\begingroup$ I don't follow this. Can you say more about your situation & your data? How are they non-independent? $\endgroup$ Aug 8, 2015 at 16:05
  • $\begingroup$ I guess I can not pull together the measurements of one subject with the measurements of another subject. $\endgroup$
    – costa
    Aug 8, 2015 at 16:10

1 Answer 1


I think you are making this too complicated. With dependent/correlated data, I would simply use a mixed model or use a generalized estimating equations approach. Using either of these methods, you would account for the non-independence of the observations and could simply include a binary indicator variable for dataset 1 versus dataset 2 in your regression model. Then you simply need to test whether or not the coefficient associated with that variable is different from zero with statistical significance.

The best part about this approach, is that this is standard methodology and you don't have to go writing your own, seemingly complicated bootstrap functions.

  • 1
    $\begingroup$ ok thanks. The problem is that I don't want to do any assumption about model or about the distribution... $\endgroup$
    – costa
    Aug 8, 2015 at 20:13
  • 1
    $\begingroup$ I see. Then, I think gung is right. We'll need to know a bit more about your situation or your data. Can you expand on that a bit? Could you simply do a permutation test? $\endgroup$ Aug 8, 2015 at 21:23
  • 1
    $\begingroup$ Yes sure, permutation tests seems to me a good thing. The data is two clouds of points (blood pressure at different time points of the day for control and for placebo), for different subjects. The thing that was worrying me is how to take into account the nested nature of the data ( some patients might have higher blood pressure and then increase artificially the overall mean . What other kind of information could be useful ? . Thanks a lot for your help. $\endgroup$
    – costa
    Aug 8, 2015 at 21:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.