1
$\begingroup$

Suppose we have two biased coins $X_1,X_2$ that are possibly correlated to each other.

In each round, when both the coins are tossed, there can be four possible outcomes: $(HH,HT,TH,TT).$ Let's denote the associated probabilities by $(p_{HH},p_{HT},p_{TH},p_{TT})$.

To make a Bayesian inference, let's assume $$(p_{HH},p_{HT},p_{TH},p_{TT})\sim Dirichlet(a_{HH},a_{HT},a_{TH},a_{TT}).$$

From this prior, we can update the probabilities of outcomes based on the observed coin-tossings.

My question is what happens if only one coin is tossed in some rounds? For example, outcomes are (H,H) in round 1, (H,no tossing) in round 2, (no tossing, T) in round 3, and so on. How can we deal with this missing data cases if we still want to make a Bayesian inference?

I initially thought I could construct the Bayesian inference by expanding the outcome space into $(HH,HN,HT,NH,NN,NT,TH,TN,TT)$, where the outcome $N$ means "no tossing" or missing data. However, this doesn't seem to be right because I'm allocating some probabilities to missing data. e.g., $p_{NN}$ is positive but "no tossing" is not even an outcome of the random variable.

Moreover, suppose that the prior is given by $$(p_{HH},p_{HN},p_{HT},p_{NH},p_{NN},p_{NT},p_{TH},p_{TN},p_{TT})\sim Dirichlet(1,1,1,1,1,1,1,1,1).$$

When we observe an outcome of $(HN)$, the posterior becomes $$(p_{HH},p_{HN},p_{HT},p_{NH},p_{NN},p_{NT},p_{TH},p_{TN},p_{TT})\sim Dirichlet(1,2,1,1,1,1,1,1,1).$$

Then, the expected probability of $H$ of the first coin is $\frac{1+2+1}{1+2+1+1+1+1+1+1+1}=\frac{4}{10}$, which does not seem to be correct.

How should I deal with this missing data case in Bayesian approach using Multinomial distribution?

|cite|improve this question
$\endgroup$
  • $\begingroup$ I believe your mistake is in expanding the outcome space to iimclude missing data/not tossed and incorporating that into the prior. Depending on the assumptions you want to make on the missing data process, you can simply use the observed data likelihood. Also, it might help to think about it this way, if you don't toss the coin, your posterior probability should be the same as your prior. $\endgroup$ – jsk Aug 20 at 16:45
  • $\begingroup$ Why is it missing? Is whether or not the data is missing for coin 2 dependent on the results for coin 1? $\endgroup$ – Louis Cialdella Aug 20 at 16:49
1
$\begingroup$

In theory, your problem is easy:

  • Compute what you need conditionnaly upon you missing data

p(what I need| My Data, The missing Data)

  • Then marginalize the missing Data

p(what I need| My Data) =

Sum_The missing Data p(what I need, The missing Data| My Data) =

Sum_The missing Data p(what I need| My Data, The missing Data) p(The missing Data)

In practice, you might run into numerical trouble if the number N of missing data is too large because the sum has 2^N terms for binary data.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Thanks @Fabrice Pautot. Could you be more specific about the posterior? For example, what would be the posterior after observing (Head,No tossing) when the prior is given by $Dir(a_{HH},a_{HL},a_{LH},a_{LL})$? $\endgroup$ – Andeanlll Sep 6 at 14:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.