If I have a coin that produces head with probability p, I know there are several methods to estimate a confidence interval for p with a given confidence level.

If I also know that the coin was randomly selected from a set of coins whose distribution of p is known, I think that it is possible to improve the quality of the estimation.

I tried to find if this is a known problem and if there are known solutions, but I found nothing.

Is there anyone who knows if this is a studied problem or has any idea on how it could be possible to leverage the known distribution of p in order to improve its estimation?

  • $\begingroup$ If you know p, then you don't need to estimate it and you don't noted confidence intervals for it. $\endgroup$
    – Tim
    Jul 28 '17 at 5:35
  • 2
    $\begingroup$ what you describe is a mixed Binomial distribution: it is like a two step approach: (1) you daw a $p$ from a distribution and then (2) given the drawn value you draw from a Binomial with that success probability. You should google for mixed binomial or for lexian distribution. $\endgroup$
    – user83346
    Jul 28 '17 at 5:45
  • $\begingroup$ It very much depends on what your distribution for $p$ is. Note that in general having variability in $p$ will likely lead to wider confidence intervals or less certainty (in a bayesian framework). $\endgroup$
    – jds
    Jul 28 '17 at 16:41
  • 1
    $\begingroup$ Tim, that's true, but I don't know p, I can just toss the coin and make observations in order to estimate p. fcop, thanks for the tip. I think I have found what I need thanks to your comment: jstor.org/stable/2984516?seq=1#page_scan_tab_contents. jds, thanks for the observation. $\endgroup$
    – maxvv
    Jul 28 '17 at 16:49
  • 1
    $\begingroup$ @Tim everything except the particular value of $p$ that was drawn when you started observing heads.... $\endgroup$
    – Glen_b
    Jul 30 '17 at 10:07

I don't think either of the comments under the question quite get at what's going on here.

My understanding of it is this (and please correct me if I am wrong): a specific value for $p$ is drawn from a known distribution, but we don't observe that $p$. Instead we observe a binomial random variable with that $p$. You want to use both the information in the known distribution for $p$ as well as in the likelihood to make some inference about $p$.

On that understanding:

A random $p$ obtained from a known distribution plus some information about that specific draw of $p$ from a sample is naturally answered with Bayesian approach; the known distribution of $p$ is the prior, which you then update via the sample (specifically the posterior is proportional to the product of the likelihood and the prior).

$$f(p|X)\,\propto\, f(X|p)\,f(p)$$

For example, if the prior were beta, the posterior would also be beta (since the beta is conjugate for a binomial likelihood).

If the prior were symmetric triangular, the posterior would be piecewise beta (with two continuous pieces).

More generally we could use a variety of techniques to obtain the updated information about $p$. On this simple problem - assuming a prior that didn't work nicely with the likelihood - one option would use numerical integration to scale the product of the prior and the likelihood to a density.

  • $\begingroup$ I don't think the question is trying to get at bayesian posteriors. I think @maxvv is trying to ask about hierarchical/compound distributions. This can be very similar to the answer you provided but could also be a frequentist question. In either way, unless $f(p)$ has a specific form, simulation may be his best bet. $\endgroup$
    – jds
    Jul 28 '17 at 16:40
  • $\begingroup$ How are you going to get a confidence interval for $p$ out of an estimate of the posterior distribution of $p$? You can get a credible interval, to be sure, but that's a different animal. $\endgroup$ Jul 28 '17 at 16:42
  • $\begingroup$ Thanks Glen, I was suspecting that the Bayesian approach was the way to follow, but I am not very familiar with statistics. I have +1 you answer, but I am not reputable enough to have my vote displayed. :) $\endgroup$
    – maxvv
    Jul 28 '17 at 16:47
  • $\begingroup$ @jds the answers could be quite different so I think it's important to try to figure out what problem is being solved. If the question were about the distribution of the number of heads, then it would be a problem of compound distributions, as you say. But the question is about inference on $p$. I don't quite see how looking at that compound distribution for the number of heads gets you to inference on $p$. If you can clarify for me how you see that working, I'd be interested to see if I could improve this answer $\endgroup$
    – Glen_b
    Jul 30 '17 at 1:11
  • 1
    $\begingroup$ @Kodi You're correct that a credible interval for $p$ is not a confidence interval; I wasn't assuming the OP was absolutely fixed on having an interval with exactly the properties of a confidence interval (like the coverage guarantee) but I should still have mentioned the difference. $\endgroup$
    – Glen_b
    Jul 30 '17 at 1:13

Thanks everybody for the help. I am indeed in an easy situation, where I can approximate my prior distribution with a beta distribution

Beta($\alpha_0$ ,$\beta_0$).

Therefore, the posterior distribution is

Beta($\alpha_0$ + observed_heads , $\beta_0$ + observed_tails).

I found an answer to another question that uses a very nice example that can be applied to my case: https://stats.stackexchange.com/a/47782/44624


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.