5
$\begingroup$

Specifically, my notes claim that when calculating confidence intervals for population proportion $p$, the estimator for the population variance is given by $\frac{P_sQ_s}{n}$, where $P_s=\frac{X}{n}$ is the random variable for the proportion of successes. Thus we have $E(P_s)=p$ and Var$(P_s)=\frac{pq}{n}$. My question is, shouldn't we have $\frac{P_sQ_s}{n} \times \frac{n}{n-1}$ as the unbiased estimator instead?

Also, from my understanding, we say that $T$ is an (unbiased) estimator of the population parameter $\theta$ if we have $E(T)=\theta$. So is there any way to prove/disprove $E(\frac{P_sQ_s}{n})=\sigma^2$, where $\sigma^2$ is the population variance?

$\endgroup$
2
  • $\begingroup$ Do your notes claim this is an unbiased estimator? $\endgroup$
    – whuber
    May 2, 2020 at 18:13
  • $\begingroup$ Nope it does not, I was just wondering if it was. Thanks so much for the answer! Just saw it now, will read through it carefully. $\endgroup$
    – Hello
    May 3, 2020 at 5:34

2 Answers 2

7
$\begingroup$

Presumably $Q_s(X) = 1 - P_s(X) = (n-X)/n.$

Writing $q=1-p$, let's work out the expectation of $n^2P_s(X)Q_s(X)$ using the definition of expectation, the formula for Binomial probabilities, and the Binomial Theorem:

$$\eqalign{ E\left[n^2P_s(X)Q_s(X)\right] &= E\left[X(n-X)\right] \\ &= \sum_x \Pr(X=x)\, x(n-x) & \text{(Definition of expectation)} \\ &= \sum_{x=0}^n \binom{n}{x}p^x q^{n-x}\, x(n-x) &\text{(Binomial distribution)} \\ &=\sum_{x=0}^n \binom{n}{x}\, pq \frac{\partial^2}{\partial p\partial q} \left(p^x\,q^{n-x}\right) \\ &= pq \frac{\partial^2}{\partial p\partial q}\sum_{x=0}^n \binom{n}{x}\, p^x\,q^{n-x} & \text{(Linearity of differentiation)}\\ &= pq \frac{\partial^2}{\partial p\partial q}\left(p+q\right)^n &\text{(Binomial Theorem)}\\ &= pq\,n(n-1)(p+q)^{n-2}. }$$

(When $n=1$ or $n=0$ the result is just $0.$) Plugging in $p+q=1$ gives

$$E\left[n^2P_sQ_s\right] = n(n-1)pq$$

for all $n,$ whence for $n\gt 1,$

$$E\left[\frac{1}{n-1}\,P_s(X)Q_s(X)\right] = \frac{pq}{n}=\operatorname{Var}\left(P_s(X)\right).$$

Therefore $P_s(X)Q_s(X)/(n-1)$ is an unbiased estimator of the variance of $X/n$ (and so obviously $P_s(X)Q_s(X)/n$ is not: it is biased).

$\endgroup$
8
  • $\begingroup$ Why then isn't the unbiased estimator used? $\endgroup$
    – Jon
    Mar 17, 2021 at 17:19
  • 1
    $\begingroup$ @Jon Since the parameter $p$ is estimated by the sample mean (in an unbiased way), and that in turn determines the population variance, it is rare for anyone to need a separate estimate of the population variance in a Binomial setting or to be able to do anything useful with it that cannot already be carried out with the estimate of $p.$ $\endgroup$
    – whuber
    Mar 17, 2021 at 17:58
  • $\begingroup$ What about for example for a confidence interval for $p$? $\endgroup$
    – Jon
    Mar 17, 2021 at 18:28
  • 1
    $\begingroup$ @Jon Most of them just use the estimate of $p.$ (There are many confidence interval procedures for Binomial data: search our site for "Clopper" to find the best summaries.) Intuitively it wouldn't make much sense to employ a separate estimate of the variance that was inconsistent with the estimate of $p$--and the simple mathematical fact is that any estimate of the variance that is a (known) constant multiple of another will give the same procedure. The only thing that changes is how you describe it. $\endgroup$
    – whuber
    Mar 17, 2021 at 18:34
  • 1
    $\begingroup$ @darkgbm That's a standard technique in combinatorics. Research formal power series. $\endgroup$
    – whuber
    Oct 4, 2023 at 13:12
2
$\begingroup$

Here's an approach using the following variance formula and rule

$Var(\hat{p})=\frac{p(1-p)}{n}=E[\hat{p}^2]-E[\hat{p}]^2$

where $\hat{p}$ is the sample proportion of times an indicator variable is 1 in a simple random sample of size $n$, i.e. the mean of an indicator variable, and $p$ is the corresponding population proportion for that indicator variable.

Suppose we estimate the population variance for that indicator variable, which is $p(1-p)$ (in terms of the population proportion $p$), using the estimator $\hat{p}(1-\hat{p})$ (which uses the sample statistics only). This estimator is biased and multiplying it by $n/(n-1)$ would make it unbiased.

Proof:

$E[\hat{p}(1-\hat{p})]=E[\hat{p}-\hat{p}^2]=p-E[\hat{p}^2]$

Rearranging the variance formula and rule above, we get $E[\hat{p}^2]=p^2 + \frac{p(1-p)}{n}$. Plugging that in we get

$E[\hat{p}(1-\hat{p})]=p - p^2 - \frac{p(1-p)}{n} = \frac{n-1}{n}p(1-p)$

$\endgroup$
1
  • $\begingroup$ +1 Simple and elegant. $\endgroup$
    – whuber
    Nov 17, 2021 at 18:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.