3
$\begingroup$

I have a larger problem but have presented what I believe is a minimal example. Imagine that you are trying to determine the true probability of a potentially-biased coin landing on heads, and want to take a bayesian perspective. Our prior is hence that the probability of heads is beta(1,1) distributed. Say that we flip the coin once and we get a heads. Our posterior is now beta(2,1).

Then we flip once more, but the coin lands crooked against an object on the table. It looks like it would have landed tails, but say that we are only 70% sure that it would have landed tails (so 30% sure that it would have been heads).

Obviously the 'best' solution is to ignore and retest, but if these coin flips are limited/expensive that might not be ideal. So is there anyway to include this result even with the uncertainty? Possibilities I've considered are

  1. Ignore result, p ~ beta(2,1)
  2. Include and pretend we are certain, p ~ beta(2,2)
  3. Include with uncertainty, p ~ beta(2, 1.7)
  4. Include with uncertainty for both, p ~ beta(2.3 1.7)

Option 4 seems reasonable, but I'm worried this is a statistical golem and I'm missing something obvious. I'm trying to stay in the bayesian setting so the answer here Distribution of partially observable binominal parameter isn't sufficient. Cheers!

$\endgroup$

2 Answers 2

1
$\begingroup$

If the coin toss going wrong is just a random thing that has nothing to do with what the result of the coin toss would have been (had nothing gone wrong), then ignoring the result of this particular toss is the easiest.

Pretending that we are completely certain or that we only add a downscaled weight to the second shape parameter (options 2 and 3) ignores the possibility of the toss could have ended up as heads (i.e. it's not quite right).

Adding 0.3 and 0.7 is the right thing to do, if you truly believe that there was a 30:70 probability that the coin would have come up heads vs. tails. However, note you need to believe this no matter how unfair the coin would be in truth. Perhaps, it only looks like that conditional on the coin being fair? Let's look at an extreme example:

  • You have observed 99 heads and 0 tails
  • A coin toss goes wrong and you feel like that was 30% likely to be heads and 70% tails.

With option 4, your belief about the proportion of heads before this 100th toss was a 95% credible interval from 0.994 to 0.9997. After this toss it's 95% CrI from 0.95 to 0.998. Before this toss, what the probability that the proportion is below what is now your lower CrI limit was less than $2 \times 10^{-22}$, but now it's 0.025. You may question whether that seems quite right, but it's indeed the right update to your belief, if you really think that toss would have landed tails up with 70% probability and heads-up with 30% probability.

Another issue with option 4 is that if you keep having such "failed" coin tosses and they all favor tails over heads (in your judgement) by 70:30, then you eventually converge to believing in a probability of the coin coming up tails being 70%. Again, as above this may be the right update.

An alternative model of what is going on is that you think that if this is a fair coin, then what you saw was increasing the probability of this toss ending up tails from 50% to 70% (=increasing the log-odds from 0 to log(0.7)-log(0.3)=0.8472979). So, in that case your belief about the coin overall influences what you believe the outcome of this coin toss was. In fact, the more you learn, your opinion on this toss will change as more data comes in in the future. In that case, some simple conjugate updating rule will not work. I feat you'll have to write down an explicit model and do MCMC sampling for it, I fear. That could look like this:

  • Observed coin tosses follow $Y_i \sim \text{Binary}(\pi)$
  • Messed up tosses also follow $Z_i \sim \text{Binary}(\pi)$ for the latent (but unobserved) outcome they would have had
  • We only observe that $ P(Z_i = 1)$ is $\text{logit}^{-1}(\text{logit}(\pi)-0.847)$.

That's actually surprisingly hard to code up in Stan (my normal preferred MCMC sampler) due to the discrete latent variable, but presumably this is possible to deal with, but it's definitely a bit messy.

$\endgroup$
2
  • $\begingroup$ Thank you very much. So it seems that if you can be accurate in your uncertainty then option 4 is reasonable. However, if if for example you had just forgotten whether it was heads or tails then updating the model to beta(a+.5, b+.5) would not be reasonable. Is that correct? EDIT: Additionally, do you have any idea on what doing option 4 might be called? In case I would want to do further reading, I was trying to avoid going the MCMC route, but agree that creating the chain of dependant probabilities would probably work. $\endgroup$
    – Jabba
    May 10, 2021 at 9:09
  • $\begingroup$ I would call this "writing down the likelihood for the data generating model and summing up the likelihood contributions from the data constellations compatibile with the observed data". I think in the missing data literature, this type of method (but also the alternative I described) would be called "full information maximum likelihood". MCMC is hard to get started with and really convenient and easy, if one gets used to it (of course, hard to say if it's worth it to get started). $\endgroup$
    – Björn
    May 10, 2021 at 12:56
0
$\begingroup$

I think you are overthinking the exact prior distribution to use. If you have examined the coin and it seems symmetrical, that information alone might push you closer to $\mathsf{Beta}(2,2)$ than to $\mathsf{Beta}(1,1).$ Also, there is accumulating experimental evidence that the method of tossing may have more to do with results than do minor imperfections in the coin.

An analysis after you have the data you can afford could show how sensitive your prior distribution is to your choice of prior.

Suppose you can afford $n = 10$ of your expensive tosses of the coin and you get seven heads and three tails. Then let's look at the practical difference between priors $\mathsf{Beta}(2,1)$ and $\mathsf{Beta}(2,1.7).$ Posteriors $\mathsf{Beta}(9,4)$ [solid blue] and $\mathsf{Beta}(9,4.7)$ plotted below.

hdr = "Posterior Distn's BETA(9,4) [blue] and BETA(9,4.7)"
curve(dbeta(x, 9,4), 0, 1, col="blue", lwd=2, main=hdr)
 curve(dbeta(x, 9,4.7), add=T, col="brown", lwd=2, lty="dotted")
 abline(h=0, col="green2")

enter image description here

Respective 95% posterior credible intervals are $(0.43, 0.90)$ and $(0.40, 0.87).$ Is the difference between these two posteriors going to make a practical difference in your course of action?

qbeta(c(.025,.975), 9,4)
[1] 0.4281415 0.9007539
qbeta(c(.025,.975), 9,4.7)
[1] 0.3974857 0.8731326
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.