Suppose we have two coins $X_1$ and $X_2$. They are possibly biased and correlated coins. The heads probability of each coins is denoted by $p_1$ and $p_2$ which we don't know at the beginning. The objective is to estimate the probabilities of HH, TT, H|H, H|T or so on as well as estimating $p_1$ and $p_2$.

To begin, I assume that $$p=(p_{HH},p_{HT},p_{TH},p_{TT})\sim Dir(a_{HH},a_{HT},a_{TH},a_{TT}).$$

If experiments (tossings of two coins, in this case) run in a perfect condition, it would be easy to make a Bayesian inference out of the experiments: we just update the parameter $a$.

However, suppose that only one coins, say coin 1, is tossed and the outcome is "H". What would the posterior look like in this case?

I think one of my three attempts should be correct, but I'm not sure which one should be.

Answer 1: $X_2$ is "H" with probability $p_2$. And the expected value of $p_2$ is $\frac{a_{HH}+a_{TH}}{a_{HH}+a_{HT}+a_{TH}+a_{TT}}$. Let $q$ denote this expected value. The posterior is $$(p_{HH},p_{HT},p_{TH},p_{TT})\sim qDir(a_{HH}+1,a_{HT},a_{TH},a_{TT})+(1-q)Dir(a_{HH},a_{HT}+1,a_{TH},a_{TT})$$ beacause $(HH)$ happens with probability $q$ and $(HT)$ occurs with the other probability.

Answer 2: Maybe I can't use the expected value to marginalize the missing information. So, let $p^1\sim Dir(a_{HH}+1,a_{HT},a_{TH},a_{TT})$ and $p^2\sim Dir(a_{HH},a_{HT}+1,a_{TH},a_{TT})$. Then, $$p =_D p_{HH}p^1+p_{HT}p^2,$$ where $=_D$ denotes the equality in distribution.

Answer 3: $X_1$ is head with probability $p_{HH}+p_{HT}$, so by the Bayes rule,

\begin{eqnarray*} f(p|X_1=H)&\propto& f(X_1=H|p)f(p)\\&=&(p_{HH}+p_{HT})f(p). \end{eqnarray*}

Obviously, the two answers lead me to different posteriors.. Could anyone tell me which one should be correct?

  • $\begingroup$ yeah, that really answers my question. Thanks $\endgroup$ – Andeanlll Oct 3 '19 at 2:21

Suppose both coins are tossed once but we only observe one of them. Assume the coin we get to observe is independent of the outcome of the toss.

Let $X_1$ and $X_2$ be random variables representing the outcome for each coin. And, let $\theta = [p_{HH}, p_{HT}, p_{TH}, p_{TT}]$ be a parameter vector indicating the joint probability of each possible pair of outcomes:

$$p(x_1, x_2 \mid \theta) = \left\{ \begin{array}{cl} p_{HH} & x_1=H, x_2=H \\ p_{HT} & x_1=H, x_2=T \\ p_{TH} & x_1=T, x_2=H \\ p_{TT} & x_1=T, x_2=T \\ \end{array} \right.$$

Suppose we've only observed the first coin. Then, the goal is to compute the posterior distribution over parameters, given the observed outcome:

$$p(\theta \mid x_1) \propto p(x_1 \mid \theta) p(\theta)$$

where $p(\theta)$ is the prior (which you've indicated is Dirichlet). $p(x_1 \mid \theta)$ is the observed data likelihood, which is obtained by marginalizing the missing outcome out of the complete data likeilhood $p(x_1, x_2 \mid \theta)$:

$$p(x_1 \mid \theta) = \sum_{x_2} p(x_1, x_2 \mid \theta)$$

So, if $X_1=H$, then $p(x_1 \mid \theta) = p_{HH} + p_{HT}$, and the posterior would be:

$$p(\theta \mid x_1) \propto (p_{HH} + p_{HT}) p(\theta)$$

Alternatively, if $X_1=T$, then $p(x_1 \mid \theta) = p_{TH} + p_{TT}$, and the posterior would be:

$$p(\theta \mid x_1) \propto (p_{TH} + p_{TT}) p(\theta)$$

If I understand your notation correctly, this is consistent with your 'answer 3'. A similar procedure can be used to compute $p(\theta \mid x_2)$ if only the second coin is observed.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.