Testing hierarchical data

Question

It is easy to compare paired data, using paired t-test. But suppose this pairing is hierarchical. Below is an example:

The stem cells of two mouse strains S1 and S2 are cultured in two different culture conditions C1 and C2, having 3 replicates for each Strain/Condition (2 x 2 x 3 = 12 samples total). We have examined the expression level of N genes in all of these samples. We want to identify the genes which are significantly (p < 0.02) higher expressed in culture condition C1 in any strain.

The case was easy if we had no replicates: just pairing the identical samples in two conditions C1 and C2 and running a paired t-test. If we had replicates but no strains the problem was easier: just running a normal t-test on two population of cells cultured in C1 and C2.

But how to deal the case while our data is hierarchical?

Answer 1

Very interesting question from a theoretical perspective (further below).

From pragmatic (purposeful) perspective you are unlikely to lose any real advantage by taking the average of the 3 replicates and treating that as the observation (and doing that paired t-test). This follow’s RA Fisher’s strategy in the design of experiment for dealing with repeated agricultural experiments where he suggested the variation between replicates be ignored except for quality assessment (i.e. do not weight the average of the replicates by their standard error). There is a modern reference given here in a slightly different context in my answer here Calculate one median for data from five experimental repetitions.

From a theoretical perspective it is related to Neyman_Scott problem where there are two strategies to deal with the strain (paired parameter) or incidental parameter. One is to model the strain parameter hierarchically (as a random parameter) and the other is to somehow remove that parameter, marginally by for instance taking paired difference or conditionally but setting the parameter as equal to an observed summary and known. If your outcome is binary, the temptation would be to do a logistic regression stratified or blocked on strain - but that is known to be highly biased. The solution here is to do an exact conditional logistic regression analysis (where conditioning has removed the strain parameter). There is a comprehensive study of this for matched multiple pairs by Breslow and Day (or one of them) that shows a considerable bias until there is more than k:1 matching (k=5??). Unfortunately, I do not know what the situation is for continuous outcomes.

There is some less technical than usual but still very technical stuff on line starting on slide 16 http://www.samsi.info/sites/default/files/Keith%20O%27Rourke%20Tuesday.pdf