# Pooling samples ahead of analysis - valid?

probably a basic question here!

I am a scientist. I'm planning an experiment with cells in a dish. The cells are cheap. I can afford n = 8 biological replicates (individual wells in a multiwell plate, each containing some number of cells.)

However, the chemical test I want to do on the cells is expensive to run. I want to quantify how much of compound X the cells produce as a result of their treatment group. The test is done on the cells' broth - you draw the liquid broth off the cells, place it in a test tube, and run the test from there.

I am wondering if there is any statistical reason I can't combine the liquid broth n = 8 wells of cells into n = 4 test tubes. This would allow me to pay for only n = 4 analysis runs. Assuming I'm careful to mix exactly the same volumes of liquid from each well, it seems to me that this is equivalent to averaging every two samples. Of course, this would mean my n for any statistical tests I run on the results of the chemical test would drop from 8 to 4.

Am I missing anything here? Seems too easy.

For biologists, I'd think this is also equivalent to going from n = 8 in a 96-well plate to n = 4 in a 48-well plate. I'd do that except that we don't have any 48-well plates in stock.

Edits requested: there are 20 treatment groups. N = 8 is wells per group (so 160 wells total).

• Welcome to Cross Validated! When you talk about n = 8 versus n = 4, is n the number of total wells or the number of wells for each treatment? How many treatments are there? Please provide that information by editing the question, as comments are easy to overlook and can be deleted Also, what you describe seems more like technical replicates rather than biological replicates; see the relevant section of this perspective.
– EdM
Nov 9, 2023 at 16:25
• Added, thank you! Also, appreciate the link; this particular experiment uses all single donor primary cells, so it’s not capturing population variability, just this patient’s response. Nov 10, 2023 at 20:07

I'm going to sidestep the (very valid!) concerns raised over technical vs. biological replication - if you're working with a cell line or inbred animals one might argue that you can only ever do technical replication even.

To answer your question, comparing $$n$$ samples drawn from individual wells or $$n$$ samples that are each a pool of $$m$$ wells might be valid if you keep in mind the following:

1. you will underestimate the well-to-well variance in the second scenario by factor $$m$$.
2. the mean must be a good summary measure for your samples.

Let me demonstrate both with a simulation example in R. I'm drawing data from $$N(0,1)$$ here but as long as a normal approximation is appropriate (or you have a sufficiently large number of samples) that shouldn't matter.

# Generate N samples from separate wells
separate <- function(n=4, fun=rnorm, iters=1E6) {
matrix(fun(n*iters), ncol=n)
}

# Generate N samples, each combined from M wells
pooled <- function(n=4, m=2, fun=rnorm, iters=1E6) {
cbind(vapply(seq_len(n), \(_) rowMeans(matrix(fun(m*iters), ncol=m)), numeric(iters)))
}

set.seed(1)
obs_separate <- separate()
obs_pooled <- pooled()


We now have a million independent experiments of each condition. To illustrate my first point, let's look at the observed standard deviation in each experiment:

# Pooling reduces variance by factor M [or SD by sqrt(M)]
summary(matrixStats::rowSds(obs_separate))
>   Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
>  0.004   0.636   0.888   0.922   1.171   3.245
summary(matrixStats::rowSds(obs_pooled))
>   Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
> 0.0063  0.4498  0.6282  0.6515  0.8274  2.3219


Notice that the standard deviation is smaller by $$\sqrt2$$ (since $$m=2$$) throughout the second condition. Not shown here is the observed means which center around the same value in both conditions, but because variance is smaller the estimates will be closer to the center in the second condition. Neither of this is a problem as long as you do the same across groups, you will maintain type I error and in fact estimate the mean more accurately as we just saw:

# Make two groups to compare (both are identical, null hypothesis is true)
set.seed(1)
n <- 8
compare_separate <- separate(n=2*n)
compare_pooled <- pooled(n=2*n, pool=n)
group <- factor(rep(0:1, each=n))

mean(genefilter::rowttests(compare_separate, group)[,3] < 0.05)
> 0.0497
mean(genefilter::rowttests(compare_pooled, group)[,3] < 0.05)
> 0.0498


However, it would be wrong to compare one group drawn from the individual setup versus another drawn from the pooled!

compare_wrong <- cbind(compare_separate[,1:n], compare_pooled[,(n+1):(2*n)])
mean(genefilter::rowttests(compare_wrong, group)[,3] < 0.05)
> 0.0601


While I've exacerbated the issue a bit by increasing the number of pools, you can see this no longer maintains type I error rate exactly because you are underestimating the variance in one group but not the other.

For the second point I'll use the log-normal distribution (i.e. a random variable whose log is normally distributed). Because of its skewness the mean is not a good measure for central location, and you would commonly use the geometric mean or median instead. However, you obviously cannot pool your samples according to a geometric principle without measuring them individually first, so that will still just be a mean.

set.seed(1)
lognorm_separate <- separate(fun=rlnorm)
lognorm_pooled <- pooled(fun=rlnorm)

# Compare geometric means
summary(exp(rowMeans(log(lognorm_separate))))
>   Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
>  0.095   0.714   1.000   1.133   1.402  11.555
summary(exp(rowMeans(log(lognorm_pooled))))
>   Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
>  0.213   0.956   1.230   1.325   1.588   8.004


Oops! Pooling has biased the estimate pretty badly. This may actually be an issue because in my field (where concentrations are usually plasma concentrations of drugs) we use the log-normal distribution and not the normal distribution. It all comes down to how appropriate a normal distribution is for your samples. If your concentrations are all well enough away from 0 and the distribution is sufficiently symmetric it might be OK, but without examining your data it's impossible to tell.

• Thank you for the detailed response! This is helping me to think through the implications of pooling. I’m not super concerned about underestimating the variance. These cells are all derived from a single donor, and the number I put in a well has no physiological meaning. They are sounding more and more like technical replicates the more I think about it. The normality assumption is a valuable point. I have some past data…I can take a look and see if it appears more normal, log normal, or something else. Thank you! Nov 10, 2023 at 20:17
• The reduced variance is a feature of pooling to be sure, this only becomes a problem if you don't pool the same in all conditions. For issue #2 there might not be too much of a problem if your concentrations are all well above 0 and cells produce continuously, the unique thing about drug metabolism is that you won't measure any if it wasn't administered (unlike, say, blood glucose of which you'll always find some), and it usually shows logarithmic decay which gives rise to the log-normal. If you pool all conditions in that case you'll still see differences that are there, although with bias. Nov 10, 2023 at 22:26
• These are cytokines - there's always some, but I'm not sure if expression is well enough above zero to truly be normal. Thanks again! Nov 12, 2023 at 0:24