Is a t-test applicable here in comparing two proportions?

Question

I have 3 categories of tissues examined by microscopy (A , B and C, 12 of each). I want to test whether the proportion of tissues staining by a particular dye, differs in the three categories. Each of the 36 tissue specimens is stained and examined under the microscope. The data is collected in the form of proportion of cells staining positive for the dye in each tissue specimen.

Now, I understand that the proportions have a binomial distribution and t-test is probably not appropriate because the variance of proportions is known. But I don't know how to apply the z-test to detect the difference in proportion here, since the data collected itself is in the form of "proportion" for each specimen. Do I calculate the mean proportion ($p_A,p_B,p_C$ for each category) by averaging the recorded proportions under each observation and then apply the z-test?

As an alternative thought, am I not estimating the proportion from my study population by averaging the proportions themselves. Thus, since I am actually estimating the true population proportion from sample itself, would the assumptions of z-test hold here?

Answer 1

As @Dave already said, there's no reason to "throw away" the information on the distribution.

However, I suspect that the more important consideration here is that your data has a nested structure:

• 3 types of tissue (A, B, C) - that's fixed factor
• 12 slides nested within each tissue type, exhibiting random variation
• a number of target cells nested in each of the slides, which are either stained or not, also random factor
• (we're presumably not looking into further factors like day-to-day variance, or spatial heterogeneity in the staining of a single tissue type)

An (unpaired) t-test or z-test or binomial proportion test cannot properly describe this structure with 3 factors/levels of data hierarchy. You'd thus need to assume that the uncertainty in estimating the proportion is negligible compared to the variance between slides. This may be achievable by counting a sufficiently large number of target cells, and you'd need to justify this for the following test.
I'd say the distribution of stained proportions across the slides is unknown, since it is not only the (binomial) variance due to the finite number of evaluated target cells, but contains also other sources of variance between slides (repeatability of the staining protocol). t-test should be OK here - its p-values may be off if your proportions are all over the place across the slides, but would give you the practically important conclusion that your staining protocol doesn't work repeatably.

Nevertheless, a generalized linear (binomial) mixed model can directly deal with your data structure. This will give you estimates of the proportions stained in A, B, and C tissues, as well as an estimate of slide-to-slide repeatability and an estimate of the binomial uncertainty due to the number of evaluated cells (i.e. a quality check for your experimental design: was the number of cells that you evaluate per slide adequate)
I'd thus recommend that you consider using that for your evaluation.

Answer 2

(Not thirty minutes ago was I reading a paper that made me think about this same topic!)

The t-test is okay for testing proportions. However, proportions are a rare time where we $100\%$ know the population distribution. Sure, there are other times where we do not know the population distribution and rely on the decent robustness of the t-test to violations of the normality assumption, but with proportions, we know that each observation is Bernoulli. Consequently, there is no statistical reason not to use a test for Bernoulli data (with the caveat that doing so is one of the smaller errors that one can make, due to the robustness of the t-test, so it probably is not that big of a deal).

If you have the data to t-test proportions, you have the data to z-test proportions. To do the t-test, you have to have the mean (the proportion) and the sample size (to get the degrees of freedom in the t-distribution). From these values, you can determine how many instances there are of your $0$-case and your $1$-case, and then you can fill in the variables in all of the equations for z-testing (or other proportion tests, like Fisher’s exact test or the chi-squared test).

Answer 3

For completeness, let's first examine each test and their assumptions and understand if using a t test in place of the z test is OK.

The reason we do a t test at all is because we have to estimate the standard error of the data, which requires an estimate of the variance. Were we to do the experiment again, that estimate might change. This is known as sampling variability and that additional variability induced by estimating the variance makes the tails of the sampling distribution slightly fatter than a normal. So in short, when we need to estimate the variance we use a t test.

Note that we don't need to estimate the variance for a binomial outcome, its a function of the estimated mean. Recall that if $X$ is binomial random variable, then

$$ \operatorname{Var}(X) = np(1-p) $$

so if we have an estimate of $p$, we also have an estimate of the variance. This is not strictly true with non-binomial data.

Is there harm in using a t test in place of a z test?

My intuition says that there should be. This is because the critical value for the t-test -- especially when you have 12 replicates per condition -- will be a lot larger and hence should hurt power. But, if you simulate the test across a range of baseline probabilities and effect sizes (shown below) we see that the choice of test doesn't really impact power

Shown below, I've run each test 1000 times for each indicated value of the true outcome (p) and an effect size (deltap). In these simulations, it doesn't seem to matter much which you choose, so if you are going to choose a test I would pick the z test to be consistent with the literature and other statistical approaches.

Answer 4

Now, I understand that the proportions have a binomial distribution

Independent Bernoulli trials have a binomial distribution, but proportions in general don't. Unless you have some reason to think that the stains are independent, a binomial distribution is not appropriate. If you're hypothesizing that the stains are independent and the samples within a category have the same population parameter p (i.e. IID within a type), then you can take the (weighted, if the number of cells in each sample is different) averages for each type. If you're not hypothesizing that, then in some sense you have a sample size of $12$ and no distribution. You could do a paired test, but the power of the test would be rather small.

and t-test is probably not appropriate because the variance of proportions is known.

I don't see how. The variance for binomial tests is calculated from p, which isn't known a priori. So that indicates t-test. But if you're assuming IID and the number of cells is large, then the difference between t-test and z-test is neglible.