Suppose we have two biased coins $X_1,X_2$ that are possibly correlated to each other.

In each round, when both the coins are tossed, there can be four possible outcomes: $(HH,HT,TH,TT).$ Let's denote the associated probabilities by $(p_{HH},p_{HT},p_{TH},p_{TT})$.

To make a Bayesian inference, let's assume $$(p_{HH},p_{HT},p_{TH},p_{TT})\sim Dirichlet(a_{HH},a_{HT},a_{TH},a_{TT}).$$

From this prior, we can update the probabilities of outcomes based on the observed coin-tossings.

My question is what happens if only one coin is tossed in some rounds? For example, outcomes are (H,H) in round 1, (H,no tossing) in round 2, (no tossing, T) in round 3, and so on. How can we deal with this missing data cases if we still want to make a Bayesian inference?

I initially thought I could construct the Bayesian inference by expanding the outcome space into $(HH,HN,HT,NH,NN,NT,TH,TN,TT)$, where the outcome $N$ means "no tossing" or missing data. However, this doesn't seem to be right because I'm allocating some probabilities to missing data. e.g., $p_{NN}$ is positive but "no tossing" is not even an outcome of the random variable.

Moreover, suppose that the prior is given by $$(p_{HH},p_{HN},p_{HT},p_{NH},p_{NN},p_{NT},p_{TH},p_{TN},p_{TT})\sim Dirichlet(1,1,1,1,1,1,1,1,1).$$

When we observe an outcome of $(HN)$, the posterior becomes $$(p_{HH},p_{HN},p_{HT},p_{NH},p_{NN},p_{NT},p_{TH},p_{TN},p_{TT})\sim Dirichlet(1,2,1,1,1,1,1,1,1).$$

Then, the expected probability of $H$ of the first coin is $\frac{1+2+1}{1+2+1+1+1+1+1+1+1}=\frac{4}{10}$, which does not seem to be correct.

How should I deal with this missing data case in Bayesian approach using Multinomial distribution?

  • $\begingroup$ I believe your mistake is in expanding the outcome space to iimclude missing data/not tossed and incorporating that into the prior. Depending on the assumptions you want to make on the missing data process, you can simply use the observed data likelihood. Also, it might help to think about it this way, if you don't toss the coin, your posterior probability should be the same as your prior. $\endgroup$ – jsk Aug 20 '19 at 16:45
  • $\begingroup$ Why is it missing? Is whether or not the data is missing for coin 2 dependent on the results for coin 1? $\endgroup$ – Louis Cialdella Aug 20 '19 at 16:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.