3
$\begingroup$

I have to test if a event have a p probability of happening. I can run this event as much times I like (given it can be run by a computer). So I was searching a way to test if the probability of this event is in an acceptable range for my case.

How can I compute, given $n$ Bernoulli Trials with probability $p$ ($P(\text{Success}) = p$ and $P(\text{Failure}) = 1-p$), the probability of ($\text{no. of successes}/n$) be within a range of $[p-a, p+a]$? [1]

Example: given 100 events, whats the probability of the proportion of successes to be greater than 45% but lesser than 55% of the total number of events?

(Apologize my lack of formalization, I have little background in statistics, only studied it in my graduation, but you can point me to the concepts and theories that I need to study to solve this problem).

Edit:

Seeing now the answers I see that I poorly asked my question.

This [1] was just a way that I thought I could achieve what I want, which is:

I have an event that I expect it to have a probability $p$ of yielding success and $1-p$ probability of yielding failure. By observing this event $n$ times, I can infer the $p'$ observed probability of that event succeeding ($\text{no of success}/n$). So I want to compare $p$ against $p'$ and check if my initial guess (which is $p$) is correct, in other words, I want to be able to claim that my event has a 99% chance of having a $p \pm \alpha$ probability of succeeding.

I thought on doing a Hypothesis Test but I couldn't quite fit my problem into it.

$\endgroup$
8
  • 1
    $\begingroup$ Please clearly specify whether you want to calculate the required probability for the proportion of successes or the ratio of number of successes to failures. $\endgroup$ Jun 11, 2019 at 17:14
  • 1
    $\begingroup$ Would it be fair to say you are looking for a 99% confidence limit on the value of $p$ based on your observations? $\endgroup$
    – whuber
    Jun 11, 2019 at 18:25
  • 1
    $\begingroup$ Yes, I think that is exactly what I need. $\endgroup$ Jun 11, 2019 at 18:31
  • 1
    $\begingroup$ Fortunately, a great deal has been written about this situation: see stats.stackexchange.com/search?q=binomial+confidence+score%3A5 for a good selection. Another good keyword for finding useful posts is Clopper. $\endgroup$
    – whuber
    Jun 11, 2019 at 19:19
  • 1
    $\begingroup$ Your title now seems inconsistent with your question body, and sometimes the body text still seems to be asking about the ratio S/F and sometimes to S/n $\endgroup$
    – Glen_b
    Jun 12, 2019 at 0:07

3 Answers 3

3
$\begingroup$

Contrary to the other answer, you don't need approximations in here. If $x$ is the number of success and $n$ is the number of trials, then you are looking for

$$ \Pr\{p-a \le \tfrac{x}{n} \le p+a\} = \Pr\{np-na \le x \le np+na\} $$

The sum of $n$ independent Bernoulli trials, each with probability of success $p$, follows binomial distribution. Knowing this, you can calculate things like

$$ \Pr\{x \le np+na\} = F_X(np+na) $$

where $F_X$ is the binomial cumulative distribution function, and use it to calculate the probability of interest.

$\endgroup$
8
  • $\begingroup$ $Pr\{x≤np+na\}=F_X(np+na)$ only gives me the probability of $p′$ being less than $p+a$, so what I really need is $F_X(np+na)∗F_{np - na}(x)$ correct? $\endgroup$ Jun 11, 2019 at 18:49
  • 1
    $\begingroup$ @BrunoAndreetto no. Think of sets and law of total probability. $\endgroup$
    – Tim
    Jun 11, 2019 at 18:54
  • 1
    $\begingroup$ @BrunoAndreetto if you want to verify or discuss it, you can post solution in comment. $\endgroup$
    – Tim
    Jun 11, 2019 at 20:07
  • 1
    $\begingroup$ @BrunoAndreetto much simpler: $F_X(np+na) - F_X(np-na)$ so $\Pr(x<u) - \Pr(x<l)$. It is a cumulative probability, so if $u>l$, then $\Pr(x<u)=\Pr(x<l)+\Pr(l<x<u)$. $\endgroup$
    – Tim
    Jun 12, 2019 at 4:09
  • 1
    $\begingroup$ Also, this is not an approximation, this is exact answer, assuming binomial distribution. $\endgroup$
    – Tim
    Jun 12, 2019 at 4:12
1
$\begingroup$

Let us start from the scratch here.

Let $X_i \, i = 1,2,...,n$ be independent random variables which take value $0$ if failure occurs on the $i^{th}$ trial and $1$ if success.

Further, let the probability of success in each trial be $p$. Then each $X_i$ are identical too.

Further we know that, from the theory of statistics, $\sum\limits_{i=1}^{n}X_i \sim Bin(n, p)$

Further, let $\hat{p} = \dfrac{\sum\limits_{i=1}^{n}x_i}{n}$ be the observed sample proportion.

We need to check or infact build a $99\%$ confidence interval for $p$ based on the observed $\hat{p}$.

Now, for a very brief description of confidence intervals, it is a random interval $(a(\textbf{X}), b(\textbf{X}))$ (where $\textbf{X}$ is the random vector) with some pre-specified level of confidence say $(1-\alpha)$ such that when the sampling procedure is repeated for a large number of times, these random intervals shall contain the true parameter $100(1-\alpha)\%$ times. You can read more about confidence intervals online.

Given this description, we may now proceed to solve our problem. So we need an interval of type $(a(\textbf{X}), b(\textbf{X}))$ such that when we repeat our experiment with $n$ trials, a large number of times, these intervals will contain the true value of $p$ on an average $99\%$ of times.

There are various methods available to construct the confidence intervals, however I would proceed with the most common one, that is Pivotal Quantity Method. Again, you can read more about this online. For the purpose of this answer, a pivotal quantity is that quantity whose distribution does not depend on any parameter.

Now, since your $n(=100)$ is large and assuming that the value of $p$ does not lie in the extremes, we have by the CLT that $Z(say) = \dfrac{\hat{p} - E(\hat{p})}{\sqrt{Var(\hat{p})}} \sim N(0,1)$. Clearly $Z$ is a pivotal quantity. Further, $E(\hat{p}) = p$ and $Var(\hat{p}) = \frac{p(1-p)}{n}$. But since, $p$ is unknown, we use an estimate of $Var(\hat{p})$ instead, which is equal to $\frac{\hat{p}(1-\hat{p})}{n}$.

Now, all we need is to construct the confidence interval.

We know that since $Z \sim N(0,1)$, $P(-Z_{0.005} < Z < Z_{0.005}) = 0.99$

Where $Z_{\alpha}$ is such that $P(Z \geq Z_{\alpha}) = \alpha$ and $Z$ is the standard normal variate.

So, $P(-Z_{0.005} < \dfrac{\sqrt{n}(\hat{p} - p)}{\sqrt{\hat{p}(1-\hat{p})}} < Z_{0.005}) = 0.99$

Finally, rearrange the inequalities inside the probability to obtain a statement in terms of the parameter $p$ and by plugging in the values of observed $\hat{p}$, you get the required confidence intervals.

Note: Another answer mentioned about using cumulative distribution function of Binomial distribution to construct such intervals. I would like to add that that is also a method. You can use any.

$\endgroup$
5
  • $\begingroup$ This is a clear account of one approach. However, the size of $n$ does not directly matter: you have no basis to invoke the CLT as a reasonable approximation until both $np$ and $n(1-p)$ are reasonably large, too. $\endgroup$
    – whuber
    Jun 11, 2019 at 19:38
  • $\begingroup$ @whuber, is the edit fine? I use the classical version of CLT here that only requires the random variables to be iid. However, if it is still wrong, let me know the correct reasoning, and I will make the necessary edits. $\endgroup$ Jun 11, 2019 at 19:45
  • $\begingroup$ When $n=100$ and $p=0.01,$ say, the CLT gives a terrible approximation to the distribution. Stating that "$p$ does not lie in the extremes" isn't a terribly helpful way to qualify the analysis. Ordinarily these would be just little nit-picks, but since this is such a basic procedure and has been so extensively discussed elsewhere on the site, you really need to provide a good reason to offer a new answer. $\endgroup$
    – whuber
    Jun 11, 2019 at 19:47
  • $\begingroup$ @whuber I only saw your comment after I had submitted my answer. And the reason I even wrote this answer was because the other answer that uses normal approximation, though in some sense correct, was somewhat misleading. $\endgroup$ Jun 11, 2019 at 20:04
  • $\begingroup$ Although I found the binomial answer was more simple, I found yours to be very clear and complete, many thanks. $\endgroup$ Jun 11, 2019 at 21:44
-2
$\begingroup$

Let's $\hat{p} = \tfrac{1}{n}\sum_i x_i$ where $x_i$ is the realisation $0$ or $1$ and $n$ the number of trials.

Using the approximation of binomial to normal distribution, you have :

$$\mathbb{P}[45\%<p<55\%] = \mathbb{P}\left[\tfrac{45\%-\hat{p}}{\sqrt{\tfrac{\hat{p}(1-\hat{p})}{n}}}<\tfrac{p-\hat{p}}{\sqrt{\tfrac{\hat{p}(1-\hat{p})}{n}}}<\tfrac{55\%-\hat{p}}{\sqrt{\tfrac{\hat{p}(1-\hat{p})}{n}}}\right] = \Phi\left[\tfrac{55\%-\hat{p}}{\sqrt{\tfrac{\hat{p}(1-\hat{p})}{n}}}\right] - \Phi\left[\tfrac{45\%-\hat{p}}{\sqrt{\tfrac{\hat{p}(1-\hat{p})}{n}}}\right]$$

where $\Phi$ is the normal distribution function.

$\endgroup$
7
  • 3
    $\begingroup$ You ought to mention that your final equality is an approximation. (It would be better to use a continuity correction, too.) $\endgroup$
    – whuber
    Jun 11, 2019 at 16:22
  • $\begingroup$ The question asks about the probability of ratio of number of successes to that of failures to lie between the given interval. And not the proportion of just successes, that you have calculated. $\endgroup$ Jun 11, 2019 at 17:10
  • $\begingroup$ After his edit, I'm more convaince of the respond I gave. $\endgroup$ Jun 11, 2019 at 17:54
  • $\begingroup$ So, P[45% < p < 55%] is the probability of p being within the range [0.45, 0.55]? Say P[45% < p < 55%] = 99%, can I claim that p is between 0.45 and 0.55 with 99% certainty? $\endgroup$ Jun 11, 2019 at 18:07
  • 3
    $\begingroup$ I think you are interchanging the two quantities. $p$ is the assumed parameter for the given Binomial model. It is not a random variable. $\hat{p}$ on the other hand is a statistic based on the sample observations and is indeed random. Thus, in fact it is $\hat{p} \sim Bin(n, p)$ and not the other way round. $\endgroup$ Jun 11, 2019 at 18:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.