Difference of two sample means from the same binomial distribution

Question

Here is the question I am trying to answer: There are 4000 birds chirping in a forest and a bird either sings with a 30% probability or doesn't with an 70% probability. Each one is independent of the other and overall noise is proportional to the number of birds. The question then asks if you listen to the birds chirping in the forest at two different times, what's the probability that the noise at the second time is more than 3% larger than the noise at the first time. It is clear to me that this is a binomial distribution with n=4000 and p=0.3, and these noise levels are sample means but I don't know where to do from there. I don't know how to determine the probability that the second sample mean will be 3% higher. Would Bayes theorem help? Any help would be great!

Answer 1

As you note in a comment, the situation can be modeled by viewing each bird as a Bernoulli$(0.3)$ variable $X_i,$ $i=1,2,\ldots, 4000,$ and the total noise level is proportional to the sum of all the $X_i.$

The question cannot be answered without making several strong assumptions, of which the most important is that all the $X_i$ are independent. In that case the sum of the $X_i$ (let's call it $Y$) has a Binomial distribution with parameters $n=4000$ and $p=0.3.$ (Otherwise we would have little idea what distribution $Y$ might have.)

In such cases (characterized by both $n\,p$ and $n(1-p)$ being "large," with "large" depending on taste but typically exceeding $5$), the distribution of $Y$ is well approximated by a Normal distribution of mean $\mu=p\,n = 0.3(4000)$ and variance $p(1-p)n = 0.3(0.7)(4000).$ Let's not do the arithmetic here, and just leave the expressions general.

Another assumption you need to make is that the two observations of $Y$ are independent. Let's denote them $Y_0$ and $Y_1.$ The question asks,

What is the chance $Y_1$ is at least $3\%$ greater than $Y_0$?

The "at least" interpretation of "3% higher" is practically forced on us, because the chance the difference is exactly 3% higher will be astronomically small: it's not even an interesting question to ask. So that's a third assumption.

It helps to phrase this event mathematically as

$$Z = Y_1 - 1.03\, Y_0 \ge 0.\tag{*}$$

Why? Because (a) it reduces an apparent two-variable problem into a one-variable problem; (b) a linear combination of independent Normal variables is Normal, (greatly) simplifying the calculations; and (c) the constant of proportionality (between noise level and the $Y_i$) doesn't matter anymore: it just multiplies everything, leaving $0$ at $0.$

Probability calculations with Normal variables require knowledge of their parameters, tantamount to the mean and variance. Let's go get these.

The mean of $Z$ evidently is

$$E[Z] = E[Y_1] - 1.03E[Y_0] = (1-1.03)p(1-p)n = (-0.03)(0.3)(0.7)(4000).$$

Ordinarily, computing the variance requires knowing the covariance of the $Y_i.$ Our independence assumption sets that covariance to $0,$ giving

$$\operatorname{Var}(Z)=\operatorname{Var}(Y_1) + (-1.03)^2\operatorname{Var}(Y_0) + 2(-1.03)\operatorname{Cov}(Y_0,Y_1) = (1 + (-1.03)^2)p(1-p)(n),$$

etc. You can do the calculations and, since you now know the parameters of $Z,$ easily compute the answer $(*),$ $\Pr(Z \ge 0).$