# How to perform a Binomial Test when having replicates?

Let's assume a cell based assay, in which some cells have an irregular shape. We can calculate the proportion of irregularly shaped cells as #irregular/#all_cells.

2 experimental treatments are assayed independently with 3 replicates each. Each replicate is an independently cultured petry-dish, in which cells are counted as described after some time.

Because of the binary outcome per cell, it seems reasonable to assume a bernoulli-distributed random process.

How would I compare such proportion data? A 2-sample binomial test comes to mind, but it's unclear how to include the replicated measurements.

And related to the comparison, I'd like to plot the mean of my proportions and a confidence interval. How would I calculate the CI?

There's a related question for count data in How to calculate confidence interval for count data in R? but I suspect that proportions should be dealt with differently.

• Could you explain, what are the replicates? What happens to the cells in the dish? – Viktor Nov 27 '17 at 15:20
• I've tried to clarify the question. – Holger Brandl Nov 27 '17 at 20:21
• You are not measuring the same cell twice. Thus, you need not be concerned with adjusting for repeated measures. However, if you believe that each replication may share some common effect, then it may be worth examining the effect of each replicant or replicated system. For example, maybe 3 sets of reagents were used and you want to adjust for anomalies in the reagents and their effect on treatment. – Todd D Nov 27 '17 at 22:03

Try fitting a binomial Generalized Linear Model - in R , if you have a dataframe called DF with numbers of successes (called "irregular") and failures ("regular") , and a column for treatment/group called Treat, with one Petri dish in each row, you can then do

Mod <- glm(data = DF, cbind(irregular,regular) ~ Treat, family = "binomial")
summary(Mod)      #This prints the results, p.values and statistics.
exp(confint(Mod)) #This gives you the CIs for the different terms in the model

• Under what circumstances would the mixed-effects logistic regression approach suggested by user163778 be more appropriate? – Holger Brandl Nov 29 '17 at 12:48
• It depends on the question you are trying to answer - the binomial glm tells you how the proportion varies as a function of the treatment, whereas the logistic regression tells you how much more likely you are to be associated with one group or the other if a certain proportion is observed. I usually choose between the two depending on what I am trying to do (predict group / predict shift in proportions). – Ankur Chakravarthy Dec 17 '17 at 2:24

There are many times in basic science (and other) research designs, where experiments are replicated and, at first glance, repeated measures seem appropriate. However, most procedures designed to handle data derived from non-independent units, such as the paired t-test, require more than one observation taken on the same experimental unit. A replicated experiment or one in which numerous observations are derived from the same organism or set of conditions are not usually measurements on identical experimental units, while shared conditions do create an appropriate environment for evaluation of clustered effects. While the presence of group or cluster effects could be considered, often these designs assume that observations (e.g. cells) taken from between replications (e.g. dishes or mice) are sufficiently homogeneous to ignore potential clustering effects. Ignoring the potential for replicate-level effects by lumping the numerous between-cluster observations into one group or treating the data as repeated measures set the stage for errors in estimation of true effects of exposure/treatment.

In the OP's case, the notion of using a logistic procedure to model a binary outcome is appropriate (a cell is either irregular or non-irregular). The idea of comparing proportions is also correct and the Chi-square test or Fisher's exact test are readily available for this purpose. This is not count data, as suggested later in the question.

If the petri dishes are considered flawlessly homogenous, then no further testing is necessary and the effect of treatment on cell morphology is complete. This approach would further be supported if each time a replicate of the treated and untreated dishes were run the replication was carried out in the same incubator with exchange of position in the incubator with the same set of reagents, etc. This situation would not create repeated measures, but would create the best experimental design for isolation of treatment effects provided the control was truly a control. A less optimal design would replicate on separate days, in multiple incubators with different lots of reagents, etc. Here, we cannot be sure that experimental conditions were homogenous between dishes and treatments effects may be lost as the control of the experimental conditions are lost.

If the OP wanted to evaluate the effect of replication on experimental results, then careful consideration of the experimental approach should be undertaken (details not provided by the OP for comment) and some hypothesis as to whether replication affected the treatment effect should be generated and tested based on the experiment as executed.

As other answers suggest, the best approach is probably to proceed with a series of generalized linear models with a binomial link function. To test the hypothesis that replication determines cell irregularity, one could test for the effect of replication as a dummy-coded variable using a model similar to: (pseudo-code):

irregular_cell = factor(replication)

If the number of groups is few and measurements within groups are many, this should suffice to test the hypothesis of replication effects on cell irregularity. If the groups are many and within-group measurements are few, then an assessment of replication effects is best estimated using a Generalized Estimating Equation or Random effects model. These models could further test the association of interactions between clusters and treatment and there are many CV references for this type of work.

Ultimately, the OP could find that replication explained a significant amount of variation and would then need to reconsider the experimental design or report treatment effects with main effects and/or standard errors adjusted for replication.

S. H. Hurlbert (1984) Pseudoreplication and the design of ecological field experiments, Ecological monographs 54(2) pp. 187 - 211

• +1 for in-depth exploration of design of experiment. It is beautiful. – EngrStudent - Reinstate Monica Nov 28 '17 at 16:24

If I understand correctly, you have 2 experimental conditions. Within each condition, you have three petry-dishes and within each petry-dish, you have the cells which you're counting. Assuming I understand correctly, You need to account for the fact that you have clustering in your data (cells are nested within dish). I think you should be able to analyze your data using a mixed-effects logistic regression with experimental condition as a predictor, irregularity (0/1) as the outcome and dish as the cluster. This method should also allow you compute CI's for your proportions (accounting for clustering).

Problem Statement:

As I understand it you have 6 petri dishes. You split them into 2 groups (A,B). Each group is identically treated, and you could $N_{irregular}/ N_{total}$.

You then want to compare treatments.

So sample data might be: $$\begin{matrix} \mathbf{Dish } & \mathbf{Group} & \mathbf{N_{irregular}} & \mathbf{N_{total}} \\ 1 & \textrm{A} & 4 & 114 \\ 2 & \textrm{A} & 20 & 100 \\ 3 & \textrm{A} & 1 & 85 \\ 4 & \textrm{B} & 17 & 108 \\ 5 & \textrm{B} & 16 & 82 \\ 6 & \textrm{B} & 10 & 89 \end{matrix}$$

So how do you compare these?

Here are our numbers:

mydata <- as.data.frame(cbind(c(1,2,3,4,5,6),
c("a","a","a","b","b","b"),
c(4,20,1,17,16,10),
c(114,100,85,108,82,89)))
names(mydata) <- c("test","group","N_irr","N_tot")

mydata


The output is this:

> mydata
test group N_irr N_tot
1    1     a     4   114
2    2     a    20   100
3    3     a     1    85
4    4     b    17   108
5    5     b    16    82
6    6     b    10    89


So now our numbers are in the computer, and we can do things like use them to make plots, or to do other analyses.

I like to start with "Gross reality checks". If you ask a biologist they might tell you that the human visual cortex has been critical for survival for 3 million years, and its ability to quickly and effectively process data is exceptional. I like to hack into that, harness it, and use it for math.

Here we turn the numbers into a picture.

...now working

• Yes, this is what I meant. – Holger Brandl Nov 27 '17 at 21:20