2
$\begingroup$

I pick a random number between 0 and 1. (My pick is uniformly distributed.) This number determines how weighted a coin is towards heads. If it's 1, there's a 100% chance that the coin will land on heads. If it's a 0, there's a 0% chance that the coin will land on heads. If it's 0.75, there's a 75% chance the coin will land on heads.

You don't know what number I picked.

I flip the coin, and it lands on heads.

What's the probability that the coin was weighted towards heads?

(Another way of phrasing this is: given that the coin landed heads, what's the probability that I picked a number greater than 0.5?)

What I've done so far:

Let $W$ (W for "Win") be a discrete indicator variable that is 1 if the coin lands heads and 0 if the coin lands tails, and let $T \sim Unif(0, 1)$ ("T" for "True probability") be a continuous random variable that represents the number that I pick.

We want to know $P(T > 0.5 \mid W=1)$. We're also given that $P(W=1 \mid T=t) = t$.

So, using Bayes' Theorem we have

$$P(T > 0.5 \mid W=1) = \frac{P(W=1 \mid T > 0.5) * P(T > 0.5)}{P(W=1)}$$.

We know $P(T > 0.5) = 0.5$ because $T \sim Unif(0, 1)$. We also know that $P(W=1) = 0.5$ due to symmetry.

So this reduces the problem down to

$$P(T > 0.5 \mid W=1) = P(W=1 \mid T > 0.5)$$.

And this is where I'm stuck.

I really want to integrate as follows, but I'm not sure if this is valid or what rule would allow me to do this.

$$ \begin{equation} \begin{split} P(W=1 \mid T > {t^*}) &= \int_{t^*}^1{P(W=1 \mid T=t)}dt \\ &= \int_{t^*}^1{t}\,dt \; \text{(by the definition of $W$)} \\ &= \frac{1-{t^*}^2}{2} \; \text{(via integration)}\\ \end{split} \end{equation} $$

But this doesn't feel quite right. I think there should be a scaling factor in there, but I'm not sure.

If we continue with this, we get the solution

$$P(T > 0.5 \mid W=1) = P(W=1 \mid T > 0.5) = \frac{1-{0.5}^2}{2} = 0.375$$.

This strikes me as intuitively wrong. It seems like it should be more likely than not that the coin is weighted towards heads, given that it landed on heads.

So I'm not quite sure how to solve this.

$\endgroup$
7
  • 1
    $\begingroup$ Did you ever divide by $P(W=1)=\frac12$? $\endgroup$
    – Henry
    Jul 13, 2021 at 21:41
  • 1
    $\begingroup$ I only ask, since the posterior distribution is triangular so the answer should be $\frac34$ $\endgroup$
    – Henry
    Jul 13, 2021 at 21:46
  • $\begingroup$ @Henry I believe so. The $P(W=1) = \frac{1}{2}$ should cancel with the $P(T > 0.5) = \frac{1}{2}$ since the latter is in the numerator and the former is in the denominator $\endgroup$
    – Pro Q
    Jul 13, 2021 at 21:48
  • 1
    $\begingroup$ You should have $f(T \mid W=1)= 2t$ and $\mathbb P(T >t \mid W=1) =1-t^2$ $\endgroup$
    – Henry
    Jul 13, 2021 at 21:51
  • $\begingroup$ Conditioning on T > 0.5 is incomplete conditioning. Condition on equalities, not inequalities. $\endgroup$ Aug 9, 2021 at 12:12

3 Answers 3

1
$\begingroup$

Carrying on where you left off,

$$ P(W = 1 | T > 0.5) $$

As $T \sim Unif(0,1)$, the conditional distribution of $T|T>a$ is $Unif(a,1), a \in (0,1)$. Therefore, integrating over the possible values of $T|T>0.5$ gives us

$$ P(W = 1 | T > 0.5) = \int_{0.5}^{1} P(W=1|T = t)f_{T|T>0.5}(t) dt \\ = \int_{0.5}^{1}t2dt\\ = t^2|_{0.5}^{1} = 1 - 0.25 = 0.75 $$

EDIT: Write the joint distribution of $(W,T)$ given $T > 0.5$ as $(U,R)$, where $U \sim Ber(r)$, $R \sim Unif(0.5,1)$. Then $P(W = 1 | T > 0.5) = P(U = 1)$, and we can find this probability by integrating $R$ out of the joint density, which is an application of the law of total probability.

$\endgroup$
5
  • $\begingroup$ What rule allows us to do the integration? Is it the law of total probability? $\endgroup$
    – Pro Q
    Jul 14, 2021 at 16:46
  • 1
    $\begingroup$ @ProQ Yes, it's the law of total probability. I added a bit more detail, hopefully it's clear. $\endgroup$ Jul 15, 2021 at 4:18
  • $\begingroup$ What is the $Ber(r)$ distribution? Is that supposed to be $Bern(r)$? $\endgroup$
    – Pro Q
    Jul 19, 2021 at 22:59
  • 1
    $\begingroup$ @ProQ Yes, it's the Bernoulli distribution. Both notations are commonly used. $\endgroup$ Jul 20, 2021 at 21:08
  • $\begingroup$ For people who see this later and are confused about how to formally compute $f_{T \mid T>0.5}(t)$, see this $\endgroup$
    – Pro Q
    Aug 5, 2021 at 6:29
1
$\begingroup$

How I would do this:

  • The prior density for $T$ is $f(t)=1$ for $t \in [0,1]$

  • The likelihood is proportional to $\mathbb P(W=1\mid T=t)=t$

  • So the posterior density for $T$ given $W=1$ for $t \in [0,1]$ is $$f(t \mid W=1) = \dfrac{1 \cdot t}{\int\limits_0^1 1 \cdot s \, ds}=2t$$

  • So $$\mathbb P(T>\tfrac12 \mid W=1)= \int\limits_{1/2}^1 2t\, dt = \frac34$$

$\endgroup$
3
  • $\begingroup$ Same result as pbeta(1, 2, 1) - pbeta(0.5, 2, 1) in R $\endgroup$
    – Henry
    Jul 13, 2021 at 22:09
  • $\begingroup$ What rule allows you to compute the posterior density? $\endgroup$
    – Pro Q
    Jul 15, 2021 at 3:26
  • 1
    $\begingroup$ @ProQ a standard application of Bayes' Theorem to a prior density $f(t\mid x) = \dfrac{f(t) \, p(x \mid t)}{\int f(s) \,p(x \mid s)\, ds}$ $\endgroup$
    – Henry
    Jul 15, 2021 at 8:21
0
$\begingroup$

This answer is just to provide clarification into David Luke Thiessen's answer, because I had a hard time understanding it at first.

From the Law of Total Probability we have:

$$P(W=1) = \int_{-\infty}^\infty{P(W=1 \mid T=t)f_T(t)}dt$$

where $f_T(t)$ is the probability density function (PDF) of $T$. We know that $f_T(t)$ is just the PDF of the uniform distribution, since $T \sim Unif(0, 1)$, so $f_T(t) = 1$ in this case, within the range $[0, 1]$.

With conditioning, the Law of Total Probability becomes:

$$P(W=1 | 0.5 < T < 1) = \int_{-\infty}^\infty{P(W=1 \mid T=t)f_{T | 0.5 < T < 1}(t)}dt$$

In this case, $f_{T | 0.5 < T < 1}(t)$ is a truncated distribution.

To quote, where our truncation is from $a=0.5$ to $b=1$:

In general, you always multiply the previous PDF by a constant $\frac{1}{F(b)−F(a)}$, where $F$ is the CDF for the function. For the uniform distribution, $F(x)=x$, so $F(1)=1$ and $F(0.5)=0.5$ and the PDF is $f(x)=1$. So, the new PDF of the truncated function is indeed $\frac{1}{F(b)−F(a)}f(x)=(2)∗(1)=2$ over the range, and 0 otherwise.

So that's how we compute that $f_{T | 0.5 < T < 1}(t) = 2$ over the range $[0.5, 1]$ and is $0$ otherwise. Since it's $0$ everywhere but from $0.5$ to $1$, we can just change this in the bounds of the integral.

Also, by definition, $P(W=1 \mid T=t) = t$.

Thus, we have:

\begin{align*} P(W=1 | 0.5 < T < 1) &= \int_{-\infty}^\infty{P(W=1 \mid T=t)f_{T | 0.5 < T < 1}(t)}dt\\ &= \int_{0.5}^1{t \cdot 2}\,dt\\ &= t^2 |_{0.5}^1\\ &= 1 − 0.25\\ &=0.75\\ \end{align*}

So, if we get a heads, there is a 75% chance that the coin is weighted towards heads.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.