2
$\begingroup$

I am really struggling with this question:

Given the assumptions for t-tests and ANOVAs, should data generated by a binomial process be analyzed with them?

Obviously, no. However we ran several simulations on R and found that the data can meet the normality assumption with a larger N. However, my TA is asking me if and when (what conditions) a binomial distribution meets the homogeneity of variance assumption, and I am really not sure.

$\endgroup$
  • $\begingroup$ Welcome to the site. If you haven't, please check the tour at stats.stackexchange.com/tour I edited the question to remove 'please help me' from the title - I suppose almost all questions are looking for some kind of help. Assuming 'TA' means this is for a course, please check stats.stackexchange.com/self-study/info $\endgroup$ – Juho Kokkala Feb 15 at 19:10
  • 1
    $\begingroup$ (1) A classical approach is to 'stabilize' variances between groups by looking at transformed data: take the square root of the arcsine of the binomial proportions. (2) As in the answ by @LucasFarias, the difficulty is that unequal success probabilities implies unequal variances. You might use a Welch t-test or ANOVA, for which one does not assume equal variances. $\endgroup$ – BruceET Feb 15 at 22:20
0
$\begingroup$

Here is a brief simulation experiment illustrating the use of the Welch two-sample t test.

Suppose group 1 has 20 observations from $\mathsf{Binom}(10, .5)$ and Group 2 has 20 observations from the same binomial distribution. Then we expect testing $H_0: p_1 = p_2$ against $H_a: p_1 \ne p_2$ at the 5% level to lead to rejection only about 5% of the time. Indeed, in one try with suitably simulated data, we do not reject $H_0$ because the P-value exceeds 0.05.

set.seed(1234)
x1 = rbinom(20, 10, .5);  x2 = rbinom(20, 10, .5)
t.test(x1, x2) 

        Welch Two Sample t-test

data:  x1 and x2
t = 0.19661, df = 37.327, p-value = 0.8452
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -0.9302373  1.1302373
sample estimates:
mean of x mean of y 
     4.85      4.75 

Running the same experiment $B = 10^5$ times leads to rejection very nearly 5% of the time, as anticipated:

set.seed(215);  B = 10^5; m = 20; n = 10
pv = replicate(B, t.test(rbinom(m, n, .5), rbinom(m, n, .5))$p.val)
mean(pv <= .05)
[1] 0.04909

However, if $p_1 \ne p_2,$ then we would hope for a correspondingly high rejection probability. In fact, an analogous simulation with $p_1 = .5$ and $p_2 = .7$ leads to rejection about 98% of the time.

set.seed(216);  B = 10^5; m = 20; n = 10
pv = replicate(B, t.test(rbinom(m, n, .5), rbinom(m, n, .7))$p.val)
mean(pv <= .05)
[1] 0.9807

For smaller samples with $p_1 = .5$ and $p_2 = .8,$ the power is also quite good, with a rejection rate of about 97%.

set.seed(217);  B = 10^5; m = 8; n = 10
pv = replicate(B, t.test(rbinom(m, n, .5), rbinom(m, n, .8))$p.val)
mean(pv <= .05)
[1] 0.96643

So for the values of $p_i$ and $n_i$ used above, the Welch accommodation for unequal variances by adjusting the degrees of freedom downward from $n_1 + n_2 - 2$ seems to be working well.

You could easily run similar experiments for whatever parameters are relevant in your own work.


Addendum: Here is how to use the variance-stabilizing transformation. This example has $p_1 = p_2,$ so variances are equal and no transformation is needed, but data are the same as in my first Welch t test, so you can compare output directly with that. As always, a difficulty using a transformation is to interpret the meaning of the results on the transformed scale.

set.seed(1234)
x1 = rbinom(20, 10, .5);  x2 = rbinom(20, 10, .5)
y1 = sqrt(asin(x1/10)); y2 = sqrt(asin(x2/10))
t.test(y1, y2, var.eq=T)  # 2-sample POOLED t test

        Two Sample t-test

data:  y1 and y2
t = 0.14278, df = 38, p-value = 0.8872
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -0.08282342  0.09539338
sample estimates:
mean of x mean of y  # that is means of y1 and y2
0.7051899 0.6989049 

Also, here is a 'transformation' version of the last simulation with small sample sizes. Power is close to the 97% obtained there.

set.seed(217);  B = 10^5; m = 8; n = 10
pv = replicate(B, 
  t.test( sqrt(asin(rbinom(m,n,.5)/n)), sqrt(asin(rbinom(m,n,.8)/n)) )$p.val)
mean(pv <= .05)
[1] 0.96317
$\endgroup$
0
$\begingroup$

For a sufficiently large n, the gaussian approximation is a close approximation to binomial distribution.

As regards to uniform variance, when there are multiple populations involved it is safe to assume uniform variance if each binomial process in the different populations have n and p approximately equal.

See this for an explanation on equal variance.

$\endgroup$
  • $\begingroup$ This is right, but what matters is whether the sampling distributions of the estimates are approximately Normal, not whether the data themselves follow (conditionally) Normal distributions. It would be interesting to have some quantitative guidance concerning how close the values of $n$ and $p$ need to be in order to work with a homoscedasticity assumption. $\endgroup$ – whuber Feb 15 at 19:14
  • 1
    $\begingroup$ I have seen this rule of thumb often applied. n >= 9 * max{p/1-p,(1-p)/p} for normal approximation provided p is not close to 0 or 1. I am thinking if a simulation solution would be helpful to check homoscedasticity. $\endgroup$ – rgk Feb 15 at 19:32
0
$\begingroup$

A Binomial distribution can be represented as the sum of trials of Bernoullis. So if there are $n$ Bernoulli trials, the sum of those has a Binomial distribution with parameters $n$ and $p$. Where $p$ is the probability of success of the Bernoulli distribution.

Now, the variance of a Bernoulli random variable is given by $p(1-p)$, which means that, for a group of Bernoulli random variables, the variance will be homogeneous if the probability of success is the same for all of them.

$\endgroup$
  • $\begingroup$ This "only if" seems rather too restrictive, because it rules out many cases where ANOVA will work just fine. $\endgroup$ – whuber Feb 15 at 19:15
  • $\begingroup$ @whuber you're right, I've corrected it. $\endgroup$ – Lucas Farias Feb 15 at 19:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.