5
$\begingroup$

I have a Bernoulli process that purportedly has mean $x$ but I hypothesize that the process actually has mean $q$. How many trials are necessary to demonstrate (to some confidence $p$) that the actual mean $\bar{x}$ is $<= q$. We can assume with no loss of generality that $q < x$ (we can choose $x' = 1-x$ and $q' = 1-q$ if this isn't the case). Furthermore, we can ignore the purported $x$ since we should only need to demonstrate that the actual mean $\bar{x}$ is $<= q$.

Alternatively, if the process actually has mean $\bar{x}$, how many trials are necessary to determine (again to some confidence $p$) that $\bar{x} < x$ for some given $x$?

$\endgroup$
1

1 Answer 1

4
$\begingroup$

So let's say you have $n$ observations, summarized in a Bernoulli variable $\def\Bin{\text{Bin}} \def\arcsin{\text{arcsin}} \def\N{\mathcal{N}} X\sim \Bin(n, \pi)$, and you want to chose between $\pi = q$ and $\pi = x$; I will change the notations and use $\pi = q_1$ and $\pi = q_2$ instead, with $q_1 < q_2$.

To avoid problems with the variance of $X$ and focus on the mean, we will use a variance stabilizing transformation: let $\phi(x) = \arcsin\sqrt x$ ; we have approximately (for $n$ big enough) $$\phi(X) \sim \N\left(\phi(\pi), {1\over 4n}\right).$$

So we need to chose between $\phi(\pi) = \phi(q_1)$ and $\phi(\pi) = \phi(q_2)$. You have two different risks to control: $\alpha$, the risk of choosing $\phi(\pi) = \phi(q_2)$ while the true value is $\phi(q_1)$, and $\beta$, the risk of choosing $\phi(\pi) = \phi(q_1)$ while the true value is $\phi(q_2)$.

The natural rule of decision is to pick a threshold $s$ between $\phi(q_1)$ and $\phi(q_2)$, and chose $\phi(q_1)$ when $\phi(X) < s$; if $\phi(X) > s$ you will chose $\phi(q_2)$.

Given our choise of $s$, when $\phi(\pi) = \phi(q_1)$ we erroneously choose $\phi(q_2)$ with probability $$\alpha = Pr( \phi(X) > s)$$ Since $\phi(\pi) = \phi(q_1)$ we can observe that $\phi(X) \sim\N\left(\phi(q_1), {1\over 4n}\right)$; standard manipulations lead to $$\alpha = Pr\bigl(Z > 2\sqrt n(s - \phi(q_1))\bigr)$$ for $Z\sim\N(0,1)$. Similarly, when $\phi(\pi) = \phi(q_2)$ we erroneously choose $\phi(q_1)$ with probability $$\beta = Pr\bigl(Z < 2\sqrt n(s - \phi(q_2))\bigr).$$

Let's say both kind of mistakes are equally damaging, and you want both to be equally small. In that case, you chose $s = {1\over 2}(\phi(q_1)+\phi(q_2))$, and you have $$\alpha = \beta = Pr\bigl(Z > \sqrt n(\phi(q_2) - \phi(q_1))\bigr).$$

$\endgroup$
5
  • $\begingroup$ How do you choose/justify the variance stabilizing transformation? Given a desired p-value, say $p < .05$, or corresponding z-value (say $z = 2$) how can I solve for $n$? $\endgroup$
    – Cirdec
    Apr 9, 2015 at 21:41
  • $\begingroup$ Something seems wrong. If $\alpha = Pr(\phi(X) > s)$ then we could argue by symmetry that $\beta = Pr(\phi(X) < s)$ and then $\alpha + \beta = 1$. $\endgroup$
    – Cirdec
    Apr 9, 2015 at 22:58
  • $\begingroup$ After fixing the substitution for $s$ to obtain the correct final equation, $\alpha = \beta = Pr\bigl(Z > \sqrt n(\phi(q_2) - \phi(q_1))\bigr)$ it is easy to solve for $n$ given a desired $z$-value for $\alpha$ and $\beta$ since $Pr\bigl(Z > z\bigr) = Pr\bigl(Z > \sqrt n(\phi(q_2) - \phi(q_1))\bigr)$ and therefore $n = \bigl({z \over \phi(q_2) - \phi(q_1)}\bigr)^2$ $\endgroup$
    – Cirdec
    Apr 10, 2015 at 1:08
  • $\begingroup$ I see that you answered most of your questions by yourself! Thanks for the edit, good job. I’ll add a reference for the variance stabilizing transformation. $\endgroup$
    – Elvis
    Apr 10, 2015 at 6:30
  • $\begingroup$ I fail to find a good reference. That would deserve a new question with "arcsine square root transformation" in the title to make it easy to find through search engines. $\endgroup$
    – Elvis
    Apr 10, 2015 at 6:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.