# Coin flipping with revision of probability

Suppose two coins $$x$$ and $$y$$ have "H" heads probability $$p_x$$ and $$p_y$$. $$p_x$$ and $$p_y$$ are independently drawn according to a uniform distribution over $$[0,1]$$.

Say that we know $$p_x\geq p_y$$. So, we update using order statistics:

$$p_x$$ follows the largest order statistics, whose CDF is $$p^2_x$$ while $$p_y$$ follows the smallest order statistics, whose CDF is $$2p_y-p_y^2.$$

The joint probability of $$(p_x,p_y)$$ will be given by $$2$$ as can be found here.

My question is what happens we flipped coin $$x$$ once and the outcome is $$H$$?

Should we or shoudn't we update the distribution of $$p_x$$ and $$p_y$$ after the observation? If we should, what would be the posterior distribution?

• not sure you should use order statistics...likelihood is unchanged...change is only in the prior...your marginal posterior for $p_x$ will probably be some kind of incomplete beta function Sep 25, 2019 at 8:09
• By the prior, you mean the uniform distribution? I was thinking about a similar problem.. In the same setting, suppose somehow that $p_x$ is revealed. Then we should revise the cdf of $p_y$ in a way that it is uniformly distributed below $p_x$.. which is the distribution of the smallest order statistics when the higher one's value is given by $p_x$. Sep 25, 2019 at 8:21

Taking your prior, which is the uniform distribution, $$f(p_x,p_y)=1$$, re-normalised over $$0\leq p_y\leq p_x\leq 1$$. This is still "uniform" in the region $$p_y\leq p_x$$. So we can say

$$f(p_x, p_y|I)\propto I_{0\leq p_y\leq p_x\leq 1}$$

The likelihood of the data observed is equal to

$$f(x=H|p_x, p_y,I)=p_x$$

Now multiply them together and we have

$$f(p_x, p_y|x=H,I)\propto (p_x, p_y|I) f(x=H|p_x, p_y,I)\propto p_x I_{0\leq p_y\leq p_x\leq 1}$$

If we want the normalizing constan, $$Z$$, just integrate. in this case it is easy

$$Z=\int_0^1 \int_0^1 p_x I_{p_x\geq p_y} dp_x dp_y=\int_0^1 \int_{p_y}^1 p_x dp_x dp_y=\frac{1}{2}\int_0^1 (1-p_y^2) dp_y=\frac{1}{3}$$

So the full posterior is $$f(p_x, p_y|x=H,I)=3 p_x I_{0\leq p_y\leq p_x\leq 1}$$

To show how it's different, if we calculate means, we have...$$E(p_x|I)=\frac{2}{3}$$ and $$E(p_x|x=H,I)=\frac{3}{4}$$....similarly we aslso have...$$E(p_y|I)=\frac{1}{3}$$ and $$E(p_y|x=H,I)=\frac{3}{8}$$. So the parameters both have the posterior mass shifted upwards. This is because the evidence provides "direct indication" that $$p_x$$ is more likely to be larger. As this is an upper bound on $$p_y$$ we have to allow for larger values of this parameter - the evidence provides "indirect indication" that $$p_y$$ is larger..

hope this helps!

• Thank you for your detailed answer. Could you explain the third expression from the top a bit more? I don't get how $f(p_x, p_y|x=H,I)\propto f(p_x, p_y|I) f(x=H|p_x, p_y,I)$ is derived. Denoting $p_x,p_y=A,~"x=H"=B,~I=C$, isn't it $f(A|BC)=\frac{f(ABC)}{f(BC)}=\frac{f(AC)}{f(BC)}\frac{f(ABC)}{f(AC)}$ which is different from your expression $\frac{f(AC)}{f(C)}\frac{f(ABC)}{f(AC)}$? Sep 25, 2019 at 15:59
• Another thing is that why can't we start with the pdf of order statistics? When $p_y<p_x$ and when they are iid draws, I think the distribution of them can be represented using orders statistics.. I apologize if I'm slow. Sep 25, 2019 at 16:27
• Oh, sorry. I got your point. It's just the way how to derive the posterior when we take $I$ as given. But I don't still get why the order statistics won't work.. Sep 26, 2019 at 2:32
• the order statistics work for the prior, just not sure the posterior is an order statistic density. You could think of first generating the 2 order statistics $(p_x^{(b)},p_y^{(b)})$, and then accepting that sample with probability $p_x^{(b)}$, and rejecting it otherwise. The accepted samples have the same distribution as the posterior Sep 26, 2019 at 12:00
• Got it. Thank you so much. Your answer really helped me a lot! Sep 26, 2019 at 14:19