I am attempting to construct a teachable example of Gibbs sampling that I can also relate to how it might be used on an actual dataset and yet could also be verified analytically by students with minimal probability education. Do you know of an example with these qualities?

One that I have been considering is the example of doing Gibbs sampling to calculate agreement between two biased coins. Consider if $X$ and $Y$ are coins with $X_{heads}=1/3$ and $Y_{heads}=3/4$. We then know analytically that agreement between the coins, $A_{true}=5/12$.

However suppose we didn't know how to solve this analytically then we would sample as follows:

  1. Set $X$ arbitrarily, say $heads$.
  2. Sample $A_{true} = P(Y_{heads}|X_{heads})$ which is $3/4$. By 'sampling' we could be flipping this $Y$ coin, or choosing $U$ from a continuous uniform distribution and tallying if $U \leq 3/4$, or we could sample from a dataset of these coin flips, a $Y_{heads}$ value from a randomly selected row where we know that $x=heads$.
  3. If for instance we got $Y_{heads}$ then sample $A_{true} = P(X_{heads}|Y_{heads}) = 1/3$ using the same approach as in step 2.
  4. Repeating thousands of times should produce a tally of $A_{true}=5/12$.

Are there other examples out there that are similarly rewarding to intuition?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.