3
$\begingroup$

I have to test if a event have a p probability of happening. I can run this event as much times I like (given it can be run by a computer). So I was searching a way to test if the probability of this event is in an acceptable range for my case.

How can I compute, given $n$ Bernoulli Trials with probability $p$ ($P(\text{Success}) = p$ and $P(\text{Failure}) = 1-p$), the probability of ($\text{no. of successes}/n$) be within a range of $[p-a, p+a]$? [1]

Example: given 100 events, whats the probability of the proportion of successes to be greater than 45% but lesser than 55% of the total number of events?

(Apologize my lack of formalization, I have little background in statistics, only studied it in my graduation, but you can point me to the concepts and theories that I need to study to solve this problem).

Edit:

Seeing now the answers I see that I poorly asked my question.

This [1] was just a way that I thought I could achieve what I want, which is:

I have an event that I expect it to have a probability $p$ of yielding success and $1-p$ probability of yielding failure. By observing this event $n$ times, I can infer the $p'$ observed probability of that event succeeding ($\text{no of success}/n$). So I want to compare $p$ against $p'$ and check if my initial guess (which is $p$) is correct, in other words, I want to be able to claim that my event has a 99% chance of having a $p \pm \alpha$ probability of succeeding.

I thought on doing a Hypothesis Test but I couldn't quite fit my problem into it.

$\endgroup$
  • 1
    $\begingroup$ Please clearly specify whether you want to calculate the required probability for the proportion of successes or the ratio of number of successes to failures. $\endgroup$ – Sanket Agrawal Jun 11 '19 at 17:14
  • 1
    $\begingroup$ Would it be fair to say you are looking for a 99% confidence limit on the value of $p$ based on your observations? $\endgroup$ – whuber Jun 11 '19 at 18:25
  • 1
    $\begingroup$ Yes, I think that is exactly what I need. $\endgroup$ – Bruno Andreetto Jun 11 '19 at 18:31
  • 1
    $\begingroup$ Fortunately, a great deal has been written about this situation: see stats.stackexchange.com/search?q=binomial+confidence+score%3A5 for a good selection. Another good keyword for finding useful posts is Clopper. $\endgroup$ – whuber Jun 11 '19 at 19:19
  • 1
    $\begingroup$ Your title now seems inconsistent with your question body, and sometimes the body text still seems to be asking about the ratio S/F and sometimes to S/n $\endgroup$ – Glen_b Jun 12 '19 at 0:07
3
$\begingroup$

Contrary to the other answer, you don't need approximations in here. If $x$ is the number of success and $n$ is the number of trials, then you are looking for

$$ \Pr\{p-a \le \tfrac{x}{n} \le p+a\} = \Pr\{np-na \le x \le np+na\} $$

The sum of $n$ independent Bernoulli trials, each with probability of success $p$, follows binomial distribution. Knowing this, you can calculate things like

$$ \Pr\{x \le np+na\} = F_X(np+na) $$

where $F_X$ is the binomial cumulative distribution function, and use it to calculate the probability of interest.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ $Pr\{x≤np+na\}=F_X(np+na)$ only gives me the probability of $p′$ being less than $p+a$, so what I really need is $F_X(np+na)∗F_{np - na}(x)$ correct? $\endgroup$ – Bruno Andreetto Jun 11 '19 at 18:49
  • 1
    $\begingroup$ @BrunoAndreetto no. Think of sets and law of total probability. $\endgroup$ – Tim Jun 11 '19 at 18:54
  • 1
    $\begingroup$ @BrunoAndreetto if you want to verify or discuss it, you can post solution in comment. $\endgroup$ – Tim Jun 11 '19 at 20:07
  • 1
    $\begingroup$ @BrunoAndreetto much simpler: $F_X(np+na) - F_X(np-na)$ so $\Pr(x<u) - \Pr(x<l)$. It is a cumulative probability, so if $u>l$, then $\Pr(x<u)=\Pr(x<l)+\Pr(l<x<u)$. $\endgroup$ – Tim Jun 12 '19 at 4:09
  • 1
    $\begingroup$ Also, this is not an approximation, this is exact answer, assuming binomial distribution. $\endgroup$ – Tim Jun 12 '19 at 4:12
1
$\begingroup$

Let us start from the scratch here.

Let $X_i \, i = 1,2,...,n$ be independent random variables which take value $0$ if failure occurs on the $i^{th}$ trial and $1$ if success.

Further, let the probability of success in each trial be $p$. Then each $X_i$ are identical too.

Further we know that, from the theory of statistics, $\sum\limits_{i=1}^{n}X_i \sim Bin(n, p)$

Further, let $\hat{p} = \dfrac{\sum\limits_{i=1}^{n}x_i}{n}$ be the observed sample proportion.

We need to check or infact build a $99\%$ confidence interval for $p$ based on the observed $\hat{p}$.

Now, for a very brief description of confidence intervals, it is a random interval $(a(\textbf{X}), b(\textbf{X}))$ (where $\textbf{X}$ is the random vector) with some pre-specified level of confidence say $(1-\alpha)$ such that when the sampling procedure is repeated for a large number of times, these random intervals shall contain the true parameter $100(1-\alpha)\%$ times. You can read more about confidence intervals online.

Given this description, we may now proceed to solve our problem. So we need an interval of type $(a(\textbf{X}), b(\textbf{X}))$ such that when we repeat our experiment with $n$ trials, a large number of times, these intervals will contain the true value of $p$ on an average $99\%$ of times.

There are various methods available to construct the confidence intervals, however I would proceed with the most common one, that is Pivotal Quantity Method. Again, you can read more about this online. For the purpose of this answer, a pivotal quantity is that quantity whose distribution does not depend on any parameter.

Now, since your $n(=100)$ is large and assuming that the value of $p$ does not lie in the extremes, we have by the CLT that $Z(say) = \dfrac{\hat{p} - E(\hat{p})}{\sqrt{Var(\hat{p})}} \sim N(0,1)$. Clearly $Z$ is a pivotal quantity. Further, $E(\hat{p}) = p$ and $Var(\hat{p}) = \frac{p(1-p)}{n}$. But since, $p$ is unknown, we use an estimate of $Var(\hat{p})$ instead, which is equal to $\frac{\hat{p}(1-\hat{p})}{n}$.

Now, all we need is to construct the confidence interval.

We know that since $Z \sim N(0,1)$, $P(-Z_{0.005} < Z < Z_{0.005}) = 0.99$

Where $Z_{\alpha}$ is such that $P(Z \geq Z_{\alpha}) = \alpha$ and $Z$ is the standard normal variate.

So, $P(-Z_{0.005} < \dfrac{\sqrt{n}(\hat{p} - p)}{\sqrt{\hat{p}(1-\hat{p})}} < Z_{0.005}) = 0.99$

Finally, rearrange the inequalities inside the probability to obtain a statement in terms of the parameter $p$ and by plugging in the values of observed $\hat{p}$, you get the required confidence intervals.

Note: Another answer mentioned about using cumulative distribution function of Binomial distribution to construct such intervals. I would like to add that that is also a method. You can use any.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ This is a clear account of one approach. However, the size of $n$ does not directly matter: you have no basis to invoke the CLT as a reasonable approximation until both $np$ and $n(1-p)$ are reasonably large, too. $\endgroup$ – whuber Jun 11 '19 at 19:38
  • $\begingroup$ @whuber, is the edit fine? I use the classical version of CLT here that only requires the random variables to be iid. However, if it is still wrong, let me know the correct reasoning, and I will make the necessary edits. $\endgroup$ – Sanket Agrawal Jun 11 '19 at 19:45
  • $\begingroup$ When $n=100$ and $p=0.01,$ say, the CLT gives a terrible approximation to the distribution. Stating that "$p$ does not lie in the extremes" isn't a terribly helpful way to qualify the analysis. Ordinarily these would be just little nit-picks, but since this is such a basic procedure and has been so extensively discussed elsewhere on the site, you really need to provide a good reason to offer a new answer. $\endgroup$ – whuber Jun 11 '19 at 19:47
  • $\begingroup$ @whuber I only saw your comment after I had submitted my answer. And the reason I even wrote this answer was because the other answer that uses normal approximation, though in some sense correct, was somewhat misleading. $\endgroup$ – Sanket Agrawal Jun 11 '19 at 20:04
  • $\begingroup$ Although I found the binomial answer was more simple, I found yours to be very clear and complete, many thanks. $\endgroup$ – Bruno Andreetto Jun 11 '19 at 21:44
-2
$\begingroup$

Let's $\hat{p} = \tfrac{1}{n}\sum_i x_i$ where $x_i$ is the realisation $0$ or $1$ and $n$ the number of trials.

Using the approximation of binomial to normal distribution, you have :

$$\mathbb{P}[45\%<p<55\%] = \mathbb{P}\left[\tfrac{45\%-\hat{p}}{\sqrt{\tfrac{\hat{p}(1-\hat{p})}{n}}}<\tfrac{p-\hat{p}}{\sqrt{\tfrac{\hat{p}(1-\hat{p})}{n}}}<\tfrac{55\%-\hat{p}}{\sqrt{\tfrac{\hat{p}(1-\hat{p})}{n}}}\right] = \Phi\left[\tfrac{55\%-\hat{p}}{\sqrt{\tfrac{\hat{p}(1-\hat{p})}{n}}}\right] - \Phi\left[\tfrac{45\%-\hat{p}}{\sqrt{\tfrac{\hat{p}(1-\hat{p})}{n}}}\right]$$

where $\Phi$ is the normal distribution function.

| cite | improve this answer | |
$\endgroup$
  • 3
    $\begingroup$ You ought to mention that your final equality is an approximation. (It would be better to use a continuity correction, too.) $\endgroup$ – whuber Jun 11 '19 at 16:22
  • $\begingroup$ The question asks about the probability of ratio of number of successes to that of failures to lie between the given interval. And not the proportion of just successes, that you have calculated. $\endgroup$ – Sanket Agrawal Jun 11 '19 at 17:10
  • $\begingroup$ After his edit, I'm more convaince of the respond I gave. $\endgroup$ – Abdoul Haki Jun 11 '19 at 17:54
  • $\begingroup$ So, P[45% < p < 55%] is the probability of p being within the range [0.45, 0.55]? Say P[45% < p < 55%] = 99%, can I claim that p is between 0.45 and 0.55 with 99% certainty? $\endgroup$ – Bruno Andreetto Jun 11 '19 at 18:07
  • 3
    $\begingroup$ I think you are interchanging the two quantities. $p$ is the assumed parameter for the given Binomial model. It is not a random variable. $\hat{p}$ on the other hand is a statistic based on the sample observations and is indeed random. Thus, in fact it is $\hat{p} \sim Bin(n, p)$ and not the other way round. $\endgroup$ – Sanket Agrawal Jun 11 '19 at 18:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.