4
$\begingroup$

Suppose you flip a coin $n$ times and you get heads $x$ times where $x < n$.

By what confidence can you say that the probability of the coin showing heads during a random toss lies between $U_1$ and $U_2$ ( $0 < u_1<u_2 < 1$)?

For example if I flip a coin 500 times and I get head 240 times, what is the probability/confidence with which I can say that probability of getting heads lies between .45 and .55?

$\endgroup$
  • $\begingroup$ can you tell me which topic to read for better understanding of such problems $\endgroup$ – Aakash Nov 28 '16 at 11:56
  • $\begingroup$ I recommend the Wikipedia article on the Topic "Checking whether a coin is fair" as in en.wikipedia.org/wiki/Checking_whether_a_coin_is_fair $\endgroup$ – Nikolas Rieble Nov 28 '16 at 12:22
  • 1
    $\begingroup$ Your conflating of the (very) different terms "probability" and "confidence," as well as their multiple uses within the question (which is asking for a "probability ... [of a] probability" or maybe a "confidence ... [of a] probability") are causing your post to attract (very) different answers. Please adopt a consistent terminology and clarify what you are trying to find. $\endgroup$ – whuber Nov 28 '16 at 18:47
  • $\begingroup$ @whuber Based on OPs comment under Tim's answer it's pretty clear that they are asking about (posterior) probability of p(H) to lie in a particular interval. Aakash, I suggest you edit to remove the word "confidence" altogether; it has an established meaning in statistics and only confuses people (that's why you got two answers that do not answer your question). $\endgroup$ – amoeba says Reinstate Monica Nov 28 '16 at 18:57
  • $\begingroup$ @amoeba I found that comment ambiguous because it gives two distinct interpretations ("it's [also] possible..."). $\endgroup$ – whuber Nov 28 '16 at 18:59
5
$\begingroup$

If probability of heads is known

If probability of heads is known, then you are not asking about confidence interval (check here and here for definition and further details), but about distribution quantiles. If you toss your coin $n$ times and observe $x \le n$ heads, then the tosses can be described by a binomial distribution parametrized by the number of tosses $n$ and probability of tossing head $p$. If you asked "what is the probability of observing $x$ or more heads in $n$ tosses?", you would be asking about cumulative distribution function

$$ \Pr(X \le x) = F_X(x) = q $$

quantile function answers the opposite question, i.e. "what is the value of $x$ such that $\Pr(X \le x) = q$ for a given value $q$?",

$$ F_X^{-1}(q) = x $$

more precisely, since cumulative distribution function of a discrete distribution is a step function, we need a generalized inverse distribution function for it

$$ F_X^{-1}(q) = \inf \{\, x \in \mathbb{R}: F_X(x) \geq q \,\} $$

Binomial distribution does not have a closed-form quantile function, but most statistical software offers you to calculate it numerically (e.g. qbinom function in R). It can be also calculated "by-hand" for discrete distributions like binomial, since you can compute cumulative probabilities $q_i$ for each $x_i = 0,1,2,\dots,n $, and then simply choose such values of $q_i$ that correspond to the probabilities of interest.

If probability of heads is unknown

If, as stated later on in comments, we are dealing with hypothetical coin "whose properties are completely unknown", then this is a different problem. We would still assume binomial distribution for counts of heads $X$, but the "unknown properties" can be understood as assuming that probability of heads $p$ is a random variable. As noticed in the comments below by amoeba and Xi'an, this can be assumed as a Bayesian problem, in terms of beta-binomial model.

Since $p$ is random and can be anything, we assume for it a "uniform" prior, i.e. beta distribution with parameters $\alpha = \beta = 1$. If we toss the coin some number of times, then we can update our prior for the additional information. Since beta is a conjugate prior for binomial distribution, updating is very simple and updated parameters become $\alpha' = \alpha + \text{number of heads}$ and $\beta' = \beta + \text{number of tails}$ and the quantiles of interest can be calculated from posterior beta distribution parametrized by $\alpha'$ and $\beta'$.

The interval of interest can be easily obtained using theoretical quantiles. Two different kinds of intervals may be obtained in this scenario:

  1. interval for number of heads $x$, that can be calculated from posterior predictive distribution that in this case is a beta-binomial distribution, or

  2. interval for possible values of $p$ that can be calculated from beta distribution.

As noted above, the interval can be calculated before seeing any data from prior distribution parametrized by $\alpha = \beta = 1$, or after seeing some data and updating for it, from posterior distribution parametrized by $\alpha'$ and $\beta'$. Each time you observe new data you can update your posterior for it to make your estimate more and more precise.

$\endgroup$
  • 1
    $\begingroup$ The OP might mean confidence intervals where by probability he means confidence level. I say this because he refers an interval [0.45, 0.55]. I will give an answer assuming confidence interval is what he means. $\endgroup$ – Michael R. Chernick Nov 28 '16 at 16:09
  • 2
    $\begingroup$ This does not answer the question. Here is how I understood it (and edited to clarify accordingly). A coin was flipped 500 times and you got H 240 times. This gives ML estimate of p(H)=240/500. Question: what is the posterior probability that p(H) is between .45 and .55? $\endgroup$ – amoeba says Reinstate Monica Nov 28 '16 at 16:29
  • $\begingroup$ amoeba: The OP never mentioned Bayes. So why are you assuming he is looking for a Bayesian solution. Also no prior was mentioned. If he was intending to do a Bayesian analysis then when he mentions probability he could mean the Bayesian a posteriori probability. But if he is taking a frequentist approach he would mean confidence.. $\endgroup$ – Michael R. Chernick Nov 28 '16 at 16:36
  • 2
    $\begingroup$ that is exactly my question . I am assuming a coin whose properties are completely unknown it is possible for a biased coin where p(h)=.2 also to show 240 heads out of 500 times there is low probability of such a set of events to occur but it's possible my question is to calculate the probability k of p(h) being in an interval say (.1 to .2) given that during our experiment it showed x heads out of n times . $\endgroup$ – Aakash Nov 28 '16 at 16:54
  • $\begingroup$ @Aakash: the way you ask your question seems to indicate you want the probability to relate to $p^h$ rather than to the experiment. This is only feasible following a Bayesian approach. $\endgroup$ – Xi'an Nov 28 '16 at 18:09
0
$\begingroup$

Assuming coin flips are independent the number on heads has a binomial distribution with mean np and variance np(1-p), where n is the sample size and p is the true success probability. To construct the interval you first specify the confidence level (often taken to be 95%). You can compute the confidence interval for the exact binomial using what is called the Clopper-Pearson method. If the sample size is large such as the 500 you posed as an example then you can use the normal approximation to construct the interval. If the interval contains 0.5 you cannot reject the hypothesis that you have a fair coin. If 0.5 lies outside the interval you would conclude that the coin is biased with a significance level equal to 1 - the confidence level, where confidence level is expressed as a proportion.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.