We are interested in determining whether a basketball player is more likely to make their second shot if they made their first. We now consider two parameters for our Bernoulli distributions θ1 and θ2.These parameters will then determine the likelihood of making the second throw. θ1 is the parameter for our Bernoulli distribution which gives us the probability of making the second throw given we have won the first, and our θ2 parameter is used to give us the probability of making the second throw if we have lost the first.
Say we have two MCMC chains, acquired through Gibbs sampling methods, the two chains are sampled from the un-normalized beta distributions of our thetas, for the purpose of determining the most probable parameter of their corresponding Bernouli distributions. How now do we determine how different the two chains are and if a player is more likely to make their second shot given they have made their first.
In Kruschke's book the differnce of the two chains are taken.
If we subtract the two chains, chain1 - chain2
chain 1 : [ [ θa , p(θa) ] , [ θb , p(θb) ] ...]
chain 2 : [ [ θ1 , p(θ1) ] , [ θ2 , p(θ2) ] ...]
We would get a chain of differences, [ [ θa - θ1, p(θa) - p(θ1)] ...] What does this chain of differences allow us to do? I am under the impression that this allows you to compare whether there is a "significant" difference between the two chains, but how exactly does this work?
I understand that 'significance' testing is a problematic area in bayesian inference but how do we determine if the two samplings from our un-normalized distributions are for lack of a better word significant.