With reference to my own post here:

'binomial' probability for ordered outcomes?

In summary, in a TV game show they asked 2 contestants 5 questions each, and I observed that in a long series of episodes one particular contestant (always the same person) consistently got 5 'easy' (E) questions, whereas the other contestant (a different person each time) consistently got 5 'difficult' (D) questions.

I was thinking whether I could apply a Bayesian approach to answer the question: what is the probability that the selection of the questions is deliberate rather than random, given the observed sequence (once)?

Let $P(F)$ be the a-priori probability that the game is rigged, and $P(S)$ the probability of getting the observed, 'suspicious' sequence of questions (EEEEE-DDDDD).

For a rigged game, clearly the probability of getting the observed sequence is 1:

$P(S|F) = 1$

On the other hand, if the questions were selected at random, the observed sequence would only appear on average once every $1024$ trials:

$P(S| \bar{F}) = \frac 1 {1024}$

This can be tested in R:

out <- replicate(1000000,paste(replicate(10,sample(c("E","D"),1)),collapse=""))
1000000/sum(out == "EEEEEDDDDD")

NOTE: to be clear, as discussed in the original post, I am aware that any individual ordered sequence has exactly the same probability = $1/1024$; here I am talking about the type of sequences (i.e. the count of E and D, taking into account the assignment of the first 5 questions to one contestant, and the last 5 to the other). So a sequence like "EDEED-DDEDE" for me is of the same type as "DDEEE-DDDEE", and the group of all such sequences has a probability = $10 \cdot 10 / 1024$, thus much more frequent than the one I observed.

So, if I apply Bayes' theorem:

$P(F|S) = P(S|F) \cdot \frac {P(F)} {P(S)} = \frac {P(S|F) \cdot P(F)} {P(S|F) \cdot P(F) + P(S|\bar F) \cdot P(\bar F)} = \frac {P(F)} {P(F) + \frac 1 {1024} \cdot (1 - P(F))}$

And that's where I am stuck. I know that in these cases one needs to make an estimate of the a-priori probability $P(F)$. How could I do that in this case?

Another metric that may be of interest is the ratio between $P(F|S)$ and $P(F)$, i.e. how much more likely it is that the game is rigged, given the observed sequence, compared to the a-priori probability:

$\frac {P(F|S)} {P(F)} = \frac {1} {P(F) + \frac 1 {1024} \cdot (1 - P(F))}$

Again, this depends on $P(F)$. Taking $P(F)$ to $0$, this becomes equal to $1/P(S|\bar F)$, so it looks like it's $1024$ times more likely that the game is rigged, because we observed this very rare sequence, than it would be in general.

Do you think this makes any sense? Any advice?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.