It is easy to compare paired data, using paired t-test. But suppose this pairing is hierarchical. Below is an example:

The stem cells of two mouse strains S1 and S2 are cultured in two different culture conditions C1 and C2, having 3 replicates for each Strain/Condition (2 x 2 x 3 = 12 samples total). We have examined the expression level of N genes in all of these samples. We want to identify the genes which are significantly (p < 0.02) higher expressed in culture condition C1 in any strain.

The case was easy if we had no replicates: just pairing the identical samples in two conditions C1 and C2 and running a paired t-test. If we had replicates but no strains the problem was easier: just running a normal t-test on two population of cells cultured in C1 and C2.

But how to deal the case while our data is hierarchical?


1 Answer 1


Very interesting question from a theoretical perspective (further below).

From pragmatic (purposeful) perspective you are unlikely to lose any real advantage by taking the average of the 3 replicates and treating that as the observation (and doing that paired t-test). This follow’s RA Fisher’s strategy in the design of experiment for dealing with repeated agricultural experiments where he suggested the variation between replicates be ignored except for quality assessment (i.e. do not weight the average of the replicates by their standard error). There is a modern reference given here in a slightly different context in my answer here Calculate one median for data from five experimental repetitions.

From a theoretical perspective it is related to Neyman_Scott problem where there are two strategies to deal with the strain (paired parameter) or incidental parameter. One is to model the strain parameter hierarchically (as a random parameter) and the other is to somehow remove that parameter, marginally by for instance taking paired difference or conditionally but setting the parameter as equal to an observed summary and known. If your outcome is binary, the temptation would be to do a logistic regression stratified or blocked on strain - but that is known to be highly biased. The solution here is to do an exact conditional logistic regression analysis (where conditioning has removed the strain parameter). There is a comprehensive study of this for matched multiple pairs by Breslow and Day (or one of them) that shows a considerable bias until there is more than k:1 matching (k=5??). Unfortunately, I do not know what the situation is for continuous outcomes.

There is some less technical than usual but still very technical stuff on line starting on slide 16 http://www.samsi.info/sites/default/files/Keith%20O%27Rourke%20Tuesday.pdf

  • $\begingroup$ Thanks for your answer. I don't believe taking only the averages of repeats is a good way: Consider gene A with identical value in 3 replicates in all strain/conditions and higher expression in condition 1. Also gene B with different expression levels in replicates but the same averages as gene A. I believe the p-value for gene A should be considerably lower than B. $\endgroup$ Oct 18, 2012 at 16:32
  • $\begingroup$ Also would you please describe a bit more about your ANOVA idea? $\endgroup$ Oct 18, 2012 at 16:59
  • $\begingroup$ @Ali Sarifi: That different p_value is though based on estimates of replicate variance based on sample size of just 3 - so unlikely to be robust. For ANOVA just think of strain as being a block in a randomised block experiment but I have disclaimed knowing that it is the least wrong way to do the analysis. $\endgroup$
    – phaneron
    Oct 18, 2012 at 17:33
  • $\begingroup$ Very interesting reply. Could you give a reference for the statement that stratified logistic regression is biased? I would like to learn more about that. $\endgroup$
    – Placidia
    Oct 19, 2012 at 15:06
  • $\begingroup$ With a little bit of searching, I found Breslow and day on line iarc.fr/en/publications/pdfs-online/stat/sp32/SP32_vol1-7.pdf $\endgroup$
    – phaneron
    Oct 19, 2012 at 16:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.