# Number of trials necessary to demonstrate Bernoulli process doesn't have mean p

I have a Bernoulli process that purportedly has mean $x$ but I hypothesize that the process actually has mean $q$. How many trials are necessary to demonstrate (to some confidence $p$) that the actual mean $\bar{x}$ is $<= q$. We can assume with no loss of generality that $q < x$ (we can choose $x' = 1-x$ and $q' = 1-q$ if this isn't the case). Furthermore, we can ignore the purported $x$ since we should only need to demonstrate that the actual mean $\bar{x}$ is $<= q$.

Alternatively, if the process actually has mean $\bar{x}$, how many trials are necessary to determine (again to some confidence $p$) that $\bar{x} < x$ for some given $x$?

So let's say you have $n$ observations, summarized in a Bernoulli variable $\def\Bin{\text{Bin}} \def\arcsin{\text{arcsin}} \def\N{\mathcal{N}} X\sim \Bin(n, \pi)$, and you want to chose between $\pi = q$ and $\pi = x$; I will change the notations and use $\pi = q_1$ and $\pi = q_2$ instead, with $q_1 < q_2$.

To avoid problems with the variance of $X$ and focus on the mean, we will use a variance stabilizing transformation: let $\phi(x) = \arcsin\sqrt x$ ; we have approximately (for $n$ big enough) $$\phi(X) \sim \N\left(\phi(\pi), {1\over 4n}\right).$$

So we need to chose between $\phi(\pi) = \phi(q_1)$ and $\phi(\pi) = \phi(q_2)$. You have two different risks to control: $\alpha$, the risk of choosing $\phi(\pi) = \phi(q_2)$ while the true value is $\phi(q_1)$, and $\beta$, the risk of choosing $\phi(\pi) = \phi(q_1)$ while the true value is $\phi(q_2)$.

The natural rule of decision is to pick a threshold $s$ between $\phi(q_1)$ and $\phi(q_2)$, and chose $\phi(q_1)$ when $\phi(X) < s$; if $\phi(X) > s$ you will chose $\phi(q_2)$.

Given our choise of $s$, when $\phi(\pi) = \phi(q_1)$ we erroneously choose $\phi(q_2)$ with probability $$\alpha = Pr( \phi(X) > s)$$ Since $\phi(\pi) = \phi(q_1)$ we can observe that $\phi(X) \sim\N\left(\phi(q_1), {1\over 4n}\right)$; standard manipulations lead to $$\alpha = Pr\bigl(Z > 2\sqrt n(s - \phi(q_1))\bigr)$$ for $Z\sim\N(0,1)$. Similarly, when $\phi(\pi) = \phi(q_2)$ we erroneously choose $\phi(q_1)$ with probability $$\beta = Pr\bigl(Z < 2\sqrt n(s - \phi(q_2))\bigr).$$

Let's say both kind of mistakes are equally damaging, and you want both to be equally small. In that case, you chose $s = {1\over 2}(\phi(q_1)+\phi(q_2))$, and you have $$\alpha = \beta = Pr\bigl(Z > \sqrt n(\phi(q_2) - \phi(q_1))\bigr).$$

• How do you choose/justify the variance stabilizing transformation? Given a desired p-value, say $p < .05$, or corresponding z-value (say $z = 2$) how can I solve for $n$? Apr 9, 2015 at 21:41
• Something seems wrong. If $\alpha = Pr(\phi(X) > s)$ then we could argue by symmetry that $\beta = Pr(\phi(X) < s)$ and then $\alpha + \beta = 1$. Apr 9, 2015 at 22:58
• After fixing the substitution for $s$ to obtain the correct final equation, $\alpha = \beta = Pr\bigl(Z > \sqrt n(\phi(q_2) - \phi(q_1))\bigr)$ it is easy to solve for $n$ given a desired $z$-value for $\alpha$ and $\beta$ since $Pr\bigl(Z > z\bigr) = Pr\bigl(Z > \sqrt n(\phi(q_2) - \phi(q_1))\bigr)$ and therefore $n = \bigl({z \over \phi(q_2) - \phi(q_1)}\bigr)^2$ Apr 10, 2015 at 1:08
• I see that you answered most of your questions by yourself! Thanks for the edit, good job. I’ll add a reference for the variance stabilizing transformation. Apr 10, 2015 at 6:30
• I fail to find a good reference. That would deserve a new question with "arcsine square root transformation" in the title to make it easy to find through search engines. Apr 10, 2015 at 6:39