Standard error for the mean of a sample of binomial random variables

Question

Suppose I'm running an experiment that can have 2 outcomes, and I'm assuming that the underlying "true" distribution of the 2 outcomes is a binomial distribution with parameters $n$ and $p$: ${\rm Binomial}(n, p)$.

I can compute the standard error, $SE_X = \frac{\sigma_X}{\sqrt{n}}$, from the form of the variance of ${\rm Binomial}(n, p)$: $$ \sigma^{2}_{X} = npq$$ where $q = 1-p$. So, $\sigma_X=\sqrt{npq}$. For the standard error I get: $SE_X=\sqrt{pq}$, but I've seen somewhere that $SE_X = \sqrt{\frac{pq}{n}}$. What did I do wrong?

Answer 1

It seems like you're using $n$ twice in two different ways - both as the sample size and as the number of bernoulli trials that comprise the Binomial random variable; to eliminate any ambiguity, I'm going to use $k$ to refer to the latter.

If you have $n$ independent samples from a ${\rm Binomial}(k,p)$ distribution, the variance of their sample mean is

$$ {\rm var} \left( \frac{1}{n} \sum_{i=1}^{n} X_{i} \right) = \frac{1}{n^2} \sum_{i=1}^{n} {\rm var}( X_{i} ) = \frac{ n {\rm var}(X_{i}) }{ n^2 } = \frac{ {\rm var}(X_{i})}{n} = \frac{ k pq }{n} $$

where $q=1-p$ and $\overline{X}$ is the same mean. This follows since

(1) ${\rm var}(cX) = c^2 {\rm var}(X)$, for any random variable, $X$, and any constant $c$.

(2) the variance of a sum of independent random variables equals the sum of the variances.

The standard error of $\overline{X}$is the square root of the variance: $\sqrt{\frac{ k pq }{n}}$. Therefore,

• When $k = n$, you get the formula you pointed out: $\sqrt{pq}$

• When $k = 1$, and the Binomial variables are just bernoulli trials, you get the formula you've seen elsewhere: $\sqrt{\frac{pq }{n}}$

Answer 2

It's easy to get two binomial distributions confused:

• distribution of number of successes
• distribution of the proportion of successes

npq is the number of successes, while npq/n = pq is the ratio of successes. This results in different standard error formulas.

Answer 3

We can look at this in the following way:

Suppose we are doing an experiment where we need to toss an unbiased coin $n$ times. The overall outcome of the experiment is $Y$ which is the summation of individual tosses (say, head as 1 and tail as 0). So, for this experiment, $Y = \sum_{i=1}^n X_i$, where $X_i$ are outcomes of individual tosses.

Here, the outcome of each toss, $X_i$, follows a Bernoulli distribution and the overall outcome $Y$ follows a binomial distribution.

The complete experiment can be thought as a single sample. Thus, if we repeat the experiment, we can get another value of $Y$, which will form another sample. All possible values of $Y$ will constitute the complete population.

Coming back to the single coin toss, which follows a Bernoulli distribution, the variance is given by $pq$, where $p$ is the probability of head (success) and $q = 1 – p$.

Now, if we look at Variance of $Y$, $V(Y) = V(\sum X_i) = \sum V(X_i)$. But, for all individual Bernoulli experiments, $V(X_i) = pq$. Since there are $n$ tosses or Bernoulli trials in the experiment, $V(Y) = \sum V(X_i) = npq$. This implies that $Y$ has variance $npq$.

Now, the sample proportion is given by $\hat p = \frac Y n$, which gives the 'proportion of success or heads'. Here, $n$ is a constant as we plan to take same no of coin tosses for all the experiments in the population.

So, $V(\frac Y n) = (\frac {1}{n^2})V(Y) = (\frac {1}{n^2})(npq) = pq/n$.

So, standard error for $\hat p$ (a sample statistic) is $\sqrt{pq/n}$

Answer 4

I think there is also some confusion in the initial post between standard error and standard deviation. Standard deviation is the sqrt of the variance of a distribution; standard error is the standard deviation of the estimated mean of a sample from that distribution, i.e., the spread of the means you would observe if you did that sample infinitely many times. The former is an intrinsic property of the distribution; the latter is a measure of the quality of your estimate of a property (the mean) of the distribution. When you do an experiment of N Bernouilli trials to estimate the unknown probability of success, the uncertainty of your estimated p=k/N after seeing k successes is a standard error of the estimated proportion, sqrt(pq/N) where q=1-p. The true distribution is characterized by a parameter P, the true probability of success. If you did an infinite number of experiments with N trials each and looked at the distribution of successes, it would have mean K=P*N, variance NPQ and standard deviation sqrt(NPQ).