3
$\begingroup$

Using the standard definition of the sample proportion random variable:

We first suppose $X \sim binomial(n, p)$ is the count of "successes" in $n$ trials where each trial has probability $p$ of success. The sample proportion random variable is $\hat{p} = X/n$, the proportion of successes.

If we are given data on several trial runs, it is possible to talk about the mean and variance of $\hat{p}$.

My question is: is there a meaningful way to talk about the mean and variance of $\hat{p}$ if the number of $n$ changes from trial to trial?

For example, if we have 3 different trial runs, with:

Run 1: 5/10 successes

Run 2: 4/8 successes

Run 3: 3/6 successes

Then intuitively it makes sense that the mean of $\hat{p}$ should be $0.5$ because each run has proportion of success $0.5$. However, is there any meaningful way to say what the variance of $\hat{p}$ would be, or does this question not make sense because $n$ changes?

$\endgroup$
2
$\begingroup$

It is known that the variance of a binomial proportion is $np(1-p)$. By the central limit theorem, the standard error of the sample average of the corresponding sequence of IID Bernoulli trials is $\sqrt{p(1-p)/n}$, which would be different for each of the 3 trials.

$\endgroup$
  • 1
    $\begingroup$ You don't need the central limit theorem for any part of your answer. The standard error of the sample average follows directly from "sequence of IID Bernoulli trials" and basic properties of variance. The standard error is exact and small sample (given the assumptions), not approximate or asymptotic. Indeed, the (classical) CLT in effect takes the same facts as given, to establish the limiting distribution of the standardized mean (i.e. given that the standard error is what comes from those basic variance properties).. $\endgroup$ – Glen_b Jun 13 '16 at 5:44
  • $\begingroup$ @glen_b good point, it's an exact SD based on the sample sum that happens to agree with the SE. Using 1.96*SE error bars would be appealing to the CLT which is at least an interesting application. $\endgroup$ – AdamO Jun 13 '16 at 14:38
  • $\begingroup$ I don't think I comprehend the distinction you're making in the first sentence. The term standard error simply means the standard deviation of the sampling distribution of some statistic; it's not so much that the standard error of the statistic happens to agree with another thing, it is that other thing. $\endgroup$ – Glen_b Jun 13 '16 at 18:14
2
$\begingroup$

The variance of the estimate of $p$ depends on how you estimate it.

If you assume your trials are all from the same process -- all independent Bernoulli trials with the same probability of success per trial -- then you might do one several reasonable things, including:

  1. combine their estimates by taking a weighted average (weighting by the inverse of variance, or precision), or

  2. combine all the trials into one, larger, set of trials

It turns out that 1 and 2 result in the same estimate (i.e. they're equivalent), so they have the same standard error.

Consider 1. $\hat{p} = \sum_{i=1}^3 w_i \hat{p}_i/\sum_{i=1}^3 w_i$ where $w_i^{-1} \propto {p(1-p)}/n_i$ (we don't know $p$, but since we assumed it's the same for each, that won't complicate things even when the individual $\hat{p}_i$s differ.

$\hat{p}_1 = \frac12,\: \text{Var}(\hat{p}_1)\propto\frac{1}{10}$

$\hat{p}_2 = \frac12,\: \text{Var}(\hat{p}_2)\propto\frac{1}{8}$

$\hat{p}_3 = \frac12,\: \text{Var}(\hat{p}_3)\propto\frac{1}{6}$

The weights are (respectively) proportional to 10, 8 and 6, and so $\hat{p}= [(10+8+6)\frac12]/[10+8+6] = \frac12$. Note that since we weight each average by its own $n$, this is the same as simply adding all the trials together and treating it as one sample (which is what happens in 2).

As a result, the variance for both is most easily found by simply treating it as the situation in 2 -- we did 24 trials and had 12 successes, so $\widehat{\text{Var}}(\hat{p})=\frac12^2/24$.

If your overall estimate is obtained in some other fashion (i.e. using a different estimator, perhaps an unweighted average for example) then it would have a different estimate of variance.

These calculations work even if the sample estimates are not identical, as long as we continue to maintain that they're independent samples from the same population. So for example, if the counts were 5/10, 5/8 and 2/6 (say) which we believed were all drawn from the same process, then we'd still get $\hat{p}=12/24$ whether we count 12 successes in 24 trials or arrive at it by weighting the three estimates in proportion to their precision.

If we believe they're not from the same process, we should firstly identify what we're estimating and what estimator we're using. Once we identify the particular estimator being used we can calculate the expectation and variance of its sampling distribution under whatever assumptions we make.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.