3
$\begingroup$

probably a basic question here!

I am a scientist. I'm planning an experiment with cells in a dish. The cells are cheap. I can afford n = 8 biological replicates (individual wells in a multiwell plate, each containing some number of cells.)

However, the chemical test I want to do on the cells is expensive to run. I want to quantify how much of compound X the cells produce as a result of their treatment group. The test is done on the cells' broth - you draw the liquid broth off the cells, place it in a test tube, and run the test from there.

I am wondering if there is any statistical reason I can't combine the liquid broth n = 8 wells of cells into n = 4 test tubes. This would allow me to pay for only n = 4 analysis runs. Assuming I'm careful to mix exactly the same volumes of liquid from each well, it seems to me that this is equivalent to averaging every two samples. Of course, this would mean my n for any statistical tests I run on the results of the chemical test would drop from 8 to 4.

Am I missing anything here? Seems too easy.

For biologists, I'd think this is also equivalent to going from n = 8 in a 96-well plate to n = 4 in a 48-well plate. I'd do that except that we don't have any 48-well plates in stock.

Edits requested: there are 20 treatment groups. N = 8 is wells per group (so 160 wells total).

$\endgroup$
2
  • 1
    $\begingroup$ Welcome to Cross Validated! When you talk about n = 8 versus n = 4, is n the number of total wells or the number of wells for each treatment? How many treatments are there? Please provide that information by editing the question, as comments are easy to overlook and can be deleted Also, what you describe seems more like technical replicates rather than biological replicates; see the relevant section of this perspective. $\endgroup$
    – EdM
    Nov 9, 2023 at 16:25
  • $\begingroup$ Added, thank you! Also, appreciate the link; this particular experiment uses all single donor primary cells, so it’s not capturing population variability, just this patient’s response. $\endgroup$
    – Chelsea
    Nov 10, 2023 at 20:07

2 Answers 2

4
$\begingroup$

I'm going to sidestep the (very valid!) concerns raised over technical vs. biological replication - if you're working with a cell line or inbred animals one might argue that you can only ever do technical replication even.

To answer your question, comparing $n$ samples drawn from individual wells or $n$ samples that are each a pool of $m$ wells might be valid if you keep in mind the following:

  1. you will underestimate the well-to-well variance in the second scenario by factor $m$.
  2. the mean must be a good summary measure for your samples.

Let me demonstrate both with a simulation example in R. I'm drawing data from $N(0,1)$ here but as long as a normal approximation is appropriate (or you have a sufficiently large number of samples) that shouldn't matter.

# Generate N samples from separate wells
separate <- function(n=4, fun=rnorm, iters=1E6) {
  matrix(fun(n*iters), ncol=n)
}

# Generate N samples, each combined from M wells
pooled <- function(n=4, m=2, fun=rnorm, iters=1E6) {
  cbind(vapply(seq_len(n), \(_) rowMeans(matrix(fun(m*iters), ncol=m)), numeric(iters)))
}

set.seed(1)
obs_separate <- separate()
obs_pooled <- pooled()

We now have a million independent experiments of each condition. To illustrate my first point, let's look at the observed standard deviation in each experiment:

# Pooling reduces variance by factor M [or SD by sqrt(M)]
summary(matrixStats::rowSds(obs_separate))
>   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
>  0.004   0.636   0.888   0.922   1.171   3.245 
summary(matrixStats::rowSds(obs_pooled))
>   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
> 0.0063  0.4498  0.6282  0.6515  0.8274  2.3219

Notice that the standard deviation is smaller by $\sqrt2$ (since $m=2$) throughout the second condition. Not shown here is the observed means which center around the same value in both conditions, but because variance is smaller the estimates will be closer to the center in the second condition. Neither of this is a problem as long as you do the same across groups, you will maintain type I error and in fact estimate the mean more accurately as we just saw:

# Make two groups to compare (both are identical, null hypothesis is true)
set.seed(1)
n <- 8
compare_separate <- separate(n=2*n)
compare_pooled <- pooled(n=2*n, pool=n)
group <- factor(rep(0:1, each=n))

mean(genefilter::rowttests(compare_separate, group)[,3] < 0.05)
> 0.0497
mean(genefilter::rowttests(compare_pooled, group)[,3] < 0.05)
> 0.0498

However, it would be wrong to compare one group drawn from the individual setup versus another drawn from the pooled!

compare_wrong <- cbind(compare_separate[,1:n], compare_pooled[,(n+1):(2*n)])
mean(genefilter::rowttests(compare_wrong, group)[,3] < 0.05)
> 0.0601

While I've exacerbated the issue a bit by increasing the number of pools, you can see this no longer maintains type I error rate exactly because you are underestimating the variance in one group but not the other.

For the second point I'll use the log-normal distribution (i.e. a random variable whose log is normally distributed). Because of its skewness the mean is not a good measure for central location, and you would commonly use the geometric mean or median instead. However, you obviously cannot pool your samples according to a geometric principle without measuring them individually first, so that will still just be a mean.

set.seed(1)
lognorm_separate <- separate(fun=rlnorm)
lognorm_pooled <- pooled(fun=rlnorm)

# Compare geometric means
summary(exp(rowMeans(log(lognorm_separate))))
>   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
>  0.095   0.714   1.000   1.133   1.402  11.555
summary(exp(rowMeans(log(lognorm_pooled))))
>   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
>  0.213   0.956   1.230   1.325   1.588   8.004

Oops! Pooling has biased the estimate pretty badly. This may actually be an issue because in my field (where concentrations are usually plasma concentrations of drugs) we use the log-normal distribution and not the normal distribution. It all comes down to how appropriate a normal distribution is for your samples. If your concentrations are all well enough away from 0 and the distribution is sufficiently symmetric it might be OK, but without examining your data it's impossible to tell.

$\endgroup$
3
  • $\begingroup$ Thank you for the detailed response! This is helping me to think through the implications of pooling. I’m not super concerned about underestimating the variance. These cells are all derived from a single donor, and the number I put in a well has no physiological meaning. They are sounding more and more like technical replicates the more I think about it. The normality assumption is a valuable point. I have some past data…I can take a look and see if it appears more normal, log normal, or something else. Thank you! $\endgroup$
    – Chelsea
    Nov 10, 2023 at 20:17
  • 1
    $\begingroup$ The reduced variance is a feature of pooling to be sure, this only becomes a problem if you don't pool the same in all conditions. For issue #2 there might not be too much of a problem if your concentrations are all well above 0 and cells produce continuously, the unique thing about drug metabolism is that you won't measure any if it wasn't administered (unlike, say, blood glucose of which you'll always find some), and it usually shows logarithmic decay which gives rise to the log-normal. If you pool all conditions in that case you'll still see differences that are there, although with bias. $\endgroup$
    – PBulls
    Nov 10, 2023 at 22:26
  • $\begingroup$ These are cytokines - there's always some, but I'm not sure if expression is well enough above zero to truly be normal. Thanks again! $\endgroup$
    – Chelsea
    Nov 12, 2023 at 0:24
2
$\begingroup$

A few points beyond what @PBulls addressed (+1) in another answer.

All of the cells you are examining come from a single donor, and thus any inferences you make will be restricted to that donor. Some might thus consider all observations to be technical replicates. In their discussion of technical versus biological replicates, however, the Pollards say (page 1361):

For example, measurements on cells within one culture flask are considered to be technical replicates, and each culture flask to be a biological replicate, if the population is all cells of this type and variability between flasks is biologically important.

I'd similarly consider all measurements on a single treatment performed in a single multi-well plate to be technical rather than biological replicates. Those cells presumably were cultured at the same time, were placed in the same multi-well plate in the same incubator, and received the same treatment.

In my experience, even working with a single cell line, variability from day to day due to uncontrolled factors can be substantial. To account for such variability, you could for example consider doing the experiments on 4 separate days (2 wells per treatment per day), pooling the 2 wells per treatment collected on the same day. That would have the same cost savings as you seek for the analysis (4 versus 8 analyses per treatment) while coming closer to what might be considered biological rather than technical replicates.

$\endgroup$
2
  • $\begingroup$ Thanks, yes, that's a great point. Curious, though: since I'm not really trying to estimate the variance introduced by my particular handling of the cells, would you think it'd be better to treat experimental run as a factor with an associated coefficient (that I can then subtract out the effects of to get to the treatment effects)? (Of course, the ideal here is that my boss pays for more cells from different donors, and I replicate the experiment from scratch...) $\endgroup$
    – Chelsea
    Nov 12, 2023 at 0:27
  • $\begingroup$ @Chelsea that's what I'm suggesting: do several experimental runs and treat the runs as a factor variable to adjust for differences in baseline values. Although you might not be really trying to estimate the variance introduced by handling of cells, a reviewer of your work will typically require more than one separate experimental run per treatment, as such variance among experimental runs is common. If all the runs agree reasonably well, you might show results for one run in your report and then state that the display is "representative of 4 [or however many] independent experiments." $\endgroup$
    – EdM
    Nov 12, 2023 at 18:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.