Pooling samples ahead of analysis - valid?

Question

probably a basic question here!

I am a scientist. I'm planning an experiment with cells in a dish. The cells are cheap. I can afford n = 8 biological replicates (individual wells in a multiwell plate, each containing some number of cells.)

However, the chemical test I want to do on the cells is expensive to run. I want to quantify how much of compound X the cells produce as a result of their treatment group. The test is done on the cells' broth - you draw the liquid broth off the cells, place it in a test tube, and run the test from there.

I am wondering if there is any statistical reason I can't combine the liquid broth n = 8 wells of cells into n = 4 test tubes. This would allow me to pay for only n = 4 analysis runs. Assuming I'm careful to mix exactly the same volumes of liquid from each well, it seems to me that this is equivalent to averaging every two samples. Of course, this would mean my n for any statistical tests I run on the results of the chemical test would drop from 8 to 4.

Am I missing anything here? Seems too easy.

For biologists, I'd think this is also equivalent to going from n = 8 in a 96-well plate to n = 4 in a 48-well plate. I'd do that except that we don't have any 48-well plates in stock.

Edits requested: there are 20 treatment groups. N = 8 is wells per group (so 160 wells total).

Answer 1

I'm going to sidestep the (very valid!) concerns raised over technical vs. biological replication - if you're working with a cell line or inbred animals one might argue that you can only ever do technical replication even.

To answer your question, comparing $n$ samples drawn from individual wells or $n$ samples that are each a pool of $m$ wells might be valid if you keep in mind the following:

1. you will underestimate the well-to-well variance in the second scenario by factor $m$.
2. the mean must be a good summary measure for your samples.

Let me demonstrate both with a simulation example in R. I'm drawing data from $N(0,1)$ here but as long as a normal approximation is appropriate (or you have a sufficiently large number of samples) that shouldn't matter.

# Generate N samples from separate wells
separate <- function(n=4, fun=rnorm, iters=1E6) {
  matrix(fun(n*iters), ncol=n)
}

# Generate N samples, each combined from M wells
pooled <- function(n=4, m=2, fun=rnorm, iters=1E6) {
  cbind(vapply(seq_len(n), \(_) rowMeans(matrix(fun(m*iters), ncol=m)), numeric(iters)))
}

set.seed(1)
obs_separate <- separate()
obs_pooled <- pooled()

We now have a million independent experiments of each condition. To illustrate my first point, let's look at the observed standard deviation in each experiment:

# Pooling reduces variance by factor M [or SD by sqrt(M)]
summary(matrixStats::rowSds(obs_separate))
>   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
>  0.004   0.636   0.888   0.922   1.171   3.245 
summary(matrixStats::rowSds(obs_pooled))
>   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
> 0.0063  0.4498  0.6282  0.6515  0.8274  2.3219

Notice that the standard deviation is smaller by $\sqrt2$ (since $m=2$) throughout the second condition. Not shown here is the observed means which center around the same value in both conditions, but because variance is smaller the estimates will be closer to the center in the second condition. Neither of this is a problem as long as you do the same across groups, you will maintain type I error and in fact estimate the mean more accurately as we just saw:

# Make two groups to compare (both are identical, null hypothesis is true)
set.seed(1)
n <- 8
compare_separate <- separate(n=2*n)
compare_pooled <- pooled(n=2*n, pool=n)
group <- factor(rep(0:1, each=n))

mean(genefilter::rowttests(compare_separate, group)[,3] < 0.05)
> 0.0497
mean(genefilter::rowttests(compare_pooled, group)[,3] < 0.05)
> 0.0498

However, it would be wrong to compare one group drawn from the individual setup versus another drawn from the pooled!

compare_wrong <- cbind(compare_separate[,1:n], compare_pooled[,(n+1):(2*n)])
mean(genefilter::rowttests(compare_wrong, group)[,3] < 0.05)
> 0.0601

While I've exacerbated the issue a bit by increasing the number of pools, you can see this no longer maintains type I error rate exactly because you are underestimating the variance in one group but not the other.

For the second point I'll use the log-normal distribution (i.e. a random variable whose log is normally distributed). Because of its skewness the mean is not a good measure for central location, and you would commonly use the geometric mean or median instead. However, you obviously cannot pool your samples according to a geometric principle without measuring them individually first, so that will still just be a mean.

set.seed(1)
lognorm_separate <- separate(fun=rlnorm)
lognorm_pooled <- pooled(fun=rlnorm)

# Compare geometric means
summary(exp(rowMeans(log(lognorm_separate))))
>   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
>  0.095   0.714   1.000   1.133   1.402  11.555
summary(exp(rowMeans(log(lognorm_pooled))))
>   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
>  0.213   0.956   1.230   1.325   1.588   8.004

Oops! Pooling has biased the estimate pretty badly. This may actually be an issue because in my field (where concentrations are usually plasma concentrations of drugs) we use the log-normal distribution and not the normal distribution. It all comes down to how appropriate a normal distribution is for your samples. If your concentrations are all well enough away from 0 and the distribution is sufficiently symmetric it might be OK, but without examining your data it's impossible to tell.

Answer 2

A few points beyond what @PBulls addressed (+1) in another answer.

All of the cells you are examining come from a single donor, and thus any inferences you make will be restricted to that donor. Some might thus consider all observations to be technical replicates. In their discussion of technical versus biological replicates, however, the Pollards say (page 1361):

For example, measurements on cells within one culture flask are considered to be technical replicates, and each culture flask to be a biological replicate, if the population is all cells of this type and variability between flasks is biologically important.

I'd similarly consider all measurements on a single treatment performed in a single multi-well plate to be technical rather than biological replicates. Those cells presumably were cultured at the same time, were placed in the same multi-well plate in the same incubator, and received the same treatment.

In my experience, even working with a single cell line, variability from day to day due to uncontrolled factors can be substantial. To account for such variability, you could for example consider doing the experiments on 4 separate days (2 wells per treatment per day), pooling the 2 wells per treatment collected on the same day. That would have the same cost savings as you seek for the analysis (4 versus 8 analyses per treatment) while coming closer to what might be considered biological rather than technical replicates.