I am really struggling with this question:
Given the assumptions for t-tests and ANOVAs, should data generated by a binomial process be analyzed with them?
Obviously, no. However we ran several simulations on R and found that the data can meet the normality assumption with a larger N. However, my TA is asking me if and when (what conditions) a binomial distribution meets the homogeneity of variance assumption, and I am really not sure.
Here is a brief simulation experiment illustrating the use of the Welch two-sample t test.
Suppose group 1 has 20 observations from $\mathsf{Binom}(10, .5)$ and Group 2 has 20 observations from the same binomial distribution. Then we expect testing $H_0: p_1 = p_2$ against $H_a: p_1 \ne p_2$ at the 5% level to lead to rejection only about 5% of the time. Indeed, in one try with suitably simulated data, we do not reject $H_0$ because the P-value exceeds 0.05.
set.seed(1234)
x1 = rbinom(20, 10, .5); x2 = rbinom(20, 10, .5)
t.test(x1, x2)
Welch Two Sample t-test
data: x1 and x2
t = 0.19661, df = 37.327, p-value = 0.8452
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
-0.9302373 1.1302373
sample estimates:
mean of x mean of y
4.85 4.75
Running the same experiment $B = 10^5$ times leads to rejection very nearly 5% of the time, as anticipated:
set.seed(215); B = 10^5; m = 20; n = 10
pv = replicate(B, t.test(rbinom(m, n, .5), rbinom(m, n, .5))$p.val)
mean(pv <= .05)
[1] 0.04909
However, if $p_1 \ne p_2,$ then we would hope for a correspondingly high rejection probability. In fact, an analogous simulation with $p_1 = .5$ and $p_2 = .7$ leads to rejection about 98% of the time.
set.seed(216); B = 10^5; m = 20; n = 10
pv = replicate(B, t.test(rbinom(m, n, .5), rbinom(m, n, .7))$p.val)
mean(pv <= .05)
[1] 0.9807
For smaller samples with $p_1 = .5$ and $p_2 = .8,$ the power is also quite good, with a rejection rate of about 97%.
set.seed(217); B = 10^5; m = 8; n = 10
pv = replicate(B, t.test(rbinom(m, n, .5), rbinom(m, n, .8))$p.val)
mean(pv <= .05)
[1] 0.96643
So for the values of $p_i$ and $n_i$ used above, the Welch accommodation for unequal variances by adjusting the degrees of freedom downward from $n_1 + n_2 - 2$ seems to be working well.
You could easily run similar experiments for whatever parameters are relevant in your own work.
Addendum: Here is how to use the variance-stabilizing transformation. This example has $p_1 = p_2,$ so variances are equal and no transformation is needed, but data are the same as in my first Welch t test, so you can compare output directly with that. As always, a difficulty using a transformation is to interpret the meaning of the results on the transformed scale.
set.seed(1234)
x1 = rbinom(20, 10, .5); x2 = rbinom(20, 10, .5)
y1 = sqrt(asin(x1/10)); y2 = sqrt(asin(x2/10))
t.test(y1, y2, var.eq=T) # 2-sample POOLED t test
Two Sample t-test
data: y1 and y2
t = 0.14278, df = 38, p-value = 0.8872
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
-0.08282342 0.09539338
sample estimates:
mean of x mean of y # that is means of y1 and y2
0.7051899 0.6989049
Also, here is a 'transformation' version of the last simulation with small sample sizes. Power is close to the 97% obtained there.
set.seed(217); B = 10^5; m = 8; n = 10
pv = replicate(B,
t.test( sqrt(asin(rbinom(m,n,.5)/n)), sqrt(asin(rbinom(m,n,.8)/n)) )$p.val)
mean(pv <= .05)
[1] 0.96317
For a sufficiently large n, the gaussian approximation is a close approximation to binomial distribution.
As regards to uniform variance, when there are multiple populations involved it is safe to assume uniform variance if each binomial process in the different populations have n and p approximately equal.
See this for an explanation on equal variance.
A Binomial distribution can be represented as the sum of trials of Bernoullis. So if there are $n$ Bernoulli trials, the sum of those has a Binomial distribution with parameters $n$ and $p$. Where $p$ is the probability of success of the Bernoulli distribution.
Now, the variance of a Bernoulli random variable is given by $p(1-p)$, which means that, for a group of Bernoulli random variables, the variance will be homogeneous if the probability of success is the same for all of them.
