I work in cell biology where the field standard is now to do 3 biological replicates of experiments. However, each data point is extremely laborious to get and we receive very little statistics training. I believe some kind of nested model would be the most rigorous, but I'm also worried about how "unfamiliar" tests are perceived.

Simply, I infected human cells with bacteria for a set length of time and counted how many cells are infected at the end per image/focus. I have control wild type untreated human cells, and 5 experimental conditions where I have treated the cells with various chemical reagents. For each experiment, I counted 20 images per control/condition. I repeated the entire experiment 3 times on different days for my biological replicates. The data is quite noisy (as expected) and each replicate is not normally distributed. The graph of my data with means of each replicate as bold squares and individual data points colored by replicate as circles is below.

Graph of data with bold squares as averages of a replicate and faint circles as individual data points

If I wanted to compare the conditions to the control with an ANOVA followed by a multiple comparisons test, could I treat the comparison of the averages of my data points per replicate as normally distributed? ie can I run an ANOVA rather than a Kruskal Wallis test on the replicate means? Since the replicate data is not normal, would comparing medians be better?

Is there an alternate test that you would recommend, especially a more simple one? From my reading, I suspect some kind of nested data test or mixed model might be the most rigorous but I've not found very accessible explanations so I am unsure how best to proceed, especially in a sub field that often just runs repeated T-tests on everything. Thanks!

EDIT: I'm getting a lot of great suggestions to follow up on- what approach is likely to have the most power to detect differences? The difference seems to be meaningful biologically based on my further work, but it would be nice if my stats could match the biology.


2 Answers 2


I recommend "Empowering statistical methods for cellular and molecular biologists," Molecular Biology of the Cell, 30:1359, 2019 as a brief useful reference, written by biologists who appreciate statistical issues.

With count data like yours, Figure 2 of that reference shows that a generalized linear model (for example, a Poisson or negative-binomial model for counts) might be most appropriate. That said, with this type of design, this particular data set, and some care you might be able to accomplish what you want with something close to ANOVA.

Your 20 images for each combination of condition and biological replicate are simply technical replicates. For inference on the effects of treatments, what you care about is the variability among the biological replicates. Thus you could consider the sum of all infected cells across the 20 images for each combination of condition and biological replicate as the outcome variable. That would give you 18 net observations (3 biological replicates for each of 6 control/treatment classes). I think that's what you are getting at.

For ANOVA or related linear models to be valid, what's important is the distribution of residuals about the modeled means. In your particular data set, it looks like 100-200 counts for each of those 18 observations. With so many counts per observation and reasonable agreement among the biological replicates under each treatment, normality of residuals might not be a bad assumption, either on the counts themselves or on some transformation of counts (log or square-root transformations of counts sometimes help). In practice, a test of normality based on only 18 observations isn't going to be very useful, although you should evaluate the distribution of residuals around the modeled values anyway.

You do need to deal with the matching among biological replicates. That might be what you are thinking about with respect to a "mixed model," which could treat the biological replicates as "random effects." With only 3 biological replicates, however, you can simply treat the biological replicates as fixed effects in the model, adjusting for baseline (WT/control) differences in the number of infected cells among the biological replicates. That's a generalization of a paired t-test. (I think that in a simple case like this, a fixed-effect and random-effect treatment of the biological replicates will provide essentially the same results.)

In R, the general form of the model then would be something like:

outcome ~ bioReplicate + condition

where outcome is an outcome measure, bioReplicate is a 3-level factor keeping track of the biological replicate, and condition is a 6-level factor (probably with "control" as the baseline level). You then decide whether to use the sum of counts across the 20 images or some transformation as the outcome. If you use the sum of counts, you would also have to choose between a standard linear regression (lm()) or a generalized linear model suitable for counts (glm() with family=poisson), although with this large number of counts that latter choice probably won't matter much.

There are other approaches that don't depend on an assumption of normality, as the answer from Robert Long explains. The non-parametric Kruskal-Wallis test is a special case of a semi-parametric proportional-odds ordinal regression model, which trades off the normality assumption for an assumption about the association of the log-odds of a higher outcome with the predictor values. In a more complicated situation than yours, ordinal regression has the advantage that it can handle continuous predictors. I find that ordinal regression also more readily lends to comparisons among multiple treatments, although in this situation you could do repeated pairwise Kruskal-Wallis tests with correction for multiple comparisons.

  • $\begingroup$ With the caveat that I know nothing about biology, doesn't the description of the experiment suggest that $n = 20$ is the size / support in a Binomial distribution. I'm not sure that Poisson or Negative Binomial make sense (even though the y-axis extends to 25 which suggests that the range of observations includes values > 20.) $\endgroup$
    – dipetkov
    Commented Nov 11, 2023 at 16:32
  • 1
    $\begingroup$ @dipetkov The set-up I assumed (based on experience): for each replicate of each treatment, you have a large number of cells on the bottom of a tissue-culture dish. You take microscopic images of 20 subregions of the dish; in each image, you count how many cells are infected. Ideally, there would also be a count of the total number of cells evaluated per image (maybe on the order of 50-100) and you would use a binomial model, but that wasn't specified in the OP. It seems that only the counts of infected cells per image (between 1 and 20) are available, hence the suggestion for a count model. $\endgroup$
    – EdM
    Commented Nov 11, 2023 at 20:30
  • $\begingroup$ Thank you for the explanation -- it makes sense though I still find it a bit hard to imagine why the only values observed are 1, 2, 3, ..., 19 infected cells per image (in 20 images). $\endgroup$
    – dipetkov
    Commented Nov 11, 2023 at 21:22
  • $\begingroup$ @dipetkov each round dot presumably represents the actual count of infected cells in an individual image, not averaged over 20 images. The squares are the averages over the 20 images for each "biological replicate." $\endgroup$
    – EdM
    Commented Nov 11, 2023 at 21:41
  • 1
    $\begingroup$ @dipetkov yes each dot is an actual count of infected cells. The bacterial kinetics for what we are more specifically measuring (total infected cells resulting from an infection of an initial single bacterium) are such that we dont see values of more than 20 infected cells. The images are taken of specific biologic criteria which is why total cells, uninfected or not, per image is not relevant as long as that value is consistent between conditions. $\endgroup$ Commented Nov 12, 2023 at 4:43

ANOVA assumes that the data within each group is normally distributed and that variances are equal across groups. When these assumptions are not met, the test can be unreliable. However, ANOVA is quite robust to violations of normality when sample sizes are equal across groups and reasonably large. ANOVA might still be appropriate here, but I would suggest some other models in order to compare the results. These are listed in increasing order of complexity:

Comparing Medians:

Comparing medians with a non-parametric test such as Kruskal-Wallis.

Repeated Measures ANOVA:

Repeated measures ANOVA would be a reasonable choice if you can plausibly assume sphericity (i.e., equal variances of differences between treatment levels), or use the Greenhouse-Geisser adjustment if sphericity does not hold.

Mixed Models:

Mixed models are very useful because they incorporate both fixed effects (treatments) and random effects (variability within biological replicates). Because the data is noisy and you have repeated measures, a mixed model would allow you to account for non-independence within each replicate and perhaps produce more accurate estimates of treatment effects.

If you have difficulty with any of these models you can always ask another question here!


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.