Suppose two coins $x$ and $y$ have "H" heads probability $p_x$ and $p_y$. $p_x$ and $p_y$ are independently drawn according to a uniform distribution over $[0,1]$.

Say that we know $p_x\geq p_y$. So, we update using order statistics:

$p_x$ follows the largest order statistics, whose CDF is $$p^2_x$$ while $p_y$ follows the smallest order statistics, whose CDF is $$2p_y-p_y^2.$$

The joint probability of $(p_x,p_y)$ will be given by $2$ as can be found here.

My question is what happens we flipped coin $x$ once and the outcome is $H$?

Should we or shoudn't we update the distribution of $p_x$ and $p_y$ after the observation? If we should, what would be the posterior distribution?

  • $\begingroup$ not sure you should use order statistics...likelihood is unchanged...change is only in the prior...your marginal posterior for $p_x$ will probably be some kind of incomplete beta function $\endgroup$ Sep 25, 2019 at 8:09
  • $\begingroup$ By the prior, you mean the uniform distribution? I was thinking about a similar problem.. In the same setting, suppose somehow that $p_x$ is revealed. Then we should revise the cdf of $p_y$ in a way that it is uniformly distributed below $p_x$.. which is the distribution of the smallest order statistics when the higher one's value is given by $p_x$. $\endgroup$
    – Andeanlll
    Sep 25, 2019 at 8:21

1 Answer 1


Taking your prior, which is the uniform distribution, $f(p_x,p_y)=1$, re-normalised over $0\leq p_y\leq p_x\leq 1$. This is still "uniform" in the region $p_y\leq p_x$. So we can say

$$f(p_x, p_y|I)\propto I_{0\leq p_y\leq p_x\leq 1}$$

The likelihood of the data observed is equal to

$$f(x=H|p_x, p_y,I)=p_x$$

Now multiply them together and we have

$$f(p_x, p_y|x=H,I)\propto (p_x, p_y|I) f(x=H|p_x, p_y,I)\propto p_x I_{0\leq p_y\leq p_x\leq 1}$$

If we want the normalizing constan, $Z$, just integrate. in this case it is easy

$$Z=\int_0^1 \int_0^1 p_x I_{p_x\geq p_y} dp_x dp_y=\int_0^1 \int_{p_y}^1 p_x dp_x dp_y=\frac{1}{2}\int_0^1 (1-p_y^2) dp_y=\frac{1}{3}$$

So the full posterior is $$f(p_x, p_y|x=H,I)=3 p_x I_{0\leq p_y\leq p_x\leq 1}$$

To show how it's different, if we calculate means, we have...$E(p_x|I)=\frac{2}{3}$ and $E(p_x|x=H,I)=\frac{3}{4}$....similarly we aslso have...$E(p_y|I)=\frac{1}{3}$ and $E(p_y|x=H,I)=\frac{3}{8}$. So the parameters both have the posterior mass shifted upwards. This is because the evidence provides "direct indication" that $p_x$ is more likely to be larger. As this is an upper bound on $p_y$ we have to allow for larger values of this parameter - the evidence provides "indirect indication" that $p_y$ is larger..

hope this helps!

  • $\begingroup$ Thank you for your detailed answer. Could you explain the third expression from the top a bit more? I don't get how $f(p_x, p_y|x=H,I)\propto f(p_x, p_y|I) f(x=H|p_x, p_y,I)$ is derived. Denoting $p_x,p_y=A,~"x=H"=B,~I=C$, isn't it $f(A|BC)=\frac{f(ABC)}{f(BC)}=\frac{f(AC)}{f(BC)}\frac{f(ABC)}{f(AC)}$ which is different from your expression $\frac{f(AC)}{f(C)}\frac{f(ABC)}{f(AC)}$? $\endgroup$
    – Andeanlll
    Sep 25, 2019 at 15:59
  • $\begingroup$ Another thing is that why can't we start with the pdf of order statistics? When $p_y<p_x$ and when they are iid draws, I think the distribution of them can be represented using orders statistics.. I apologize if I'm slow. $\endgroup$
    – Andeanlll
    Sep 25, 2019 at 16:27
  • $\begingroup$ Oh, sorry. I got your point. It's just the way how to derive the posterior when we take $I$ as given. But I don't still get why the order statistics won't work.. $\endgroup$
    – Andeanlll
    Sep 26, 2019 at 2:32
  • $\begingroup$ the order statistics work for the prior, just not sure the posterior is an order statistic density. You could think of first generating the 2 order statistics $(p_x^{(b)},p_y^{(b)})$, and then accepting that sample with probability $p_x^{(b)}$, and rejecting it otherwise. The accepted samples have the same distribution as the posterior $\endgroup$ Sep 26, 2019 at 12:00
  • $\begingroup$ Got it. Thank you so much. Your answer really helped me a lot! $\endgroup$
    – Andeanlll
    Sep 26, 2019 at 14:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.