I'll try to make this as to the point as possible.
Consider 2 people playing a few games against a computer.
Player 1 gets 6/10 wins
Player 2 gets 56/100 wins
What's the chance that player 1 is better than player 2?
This is a question I've been struggling with for a few days now, and I've come across this question, which is completely equivalent:
Binomial Probability Question
However, it leaves me with more questions than I came with.
Disclaimer: It's been years since I've had an introductory course to statistics. I've tried hard to look up answers, but most of it is going over my head. (I knew nothing about Bayesian statistics yesterday, and just a little now.)
Questions:
- In the link I provided, they use Bayesian statistics. Is there a reasonable frequentist way to solve this problem? (I might understand such an approach better)
I tried to find a Probability Density Function for the "true" win percentage of each player. A section in my old textbook, Confidence interval for a proportion, looked promising. However, this relies on the fact that the samples need to be sufficiently large, which is not the case here.
- In the linked answer, the integrals come a bit out of nowhere (at least for me). Could someone explain how they came to the posterior probability distribution in that first expression?
Also, the last step
We then need to integrate that over the probability distribution for John's free throw percentages.
seems to be the key. I think I have a general understanding of what is happening, but if a nice (graphical) explanation of this is available online, I'd also be really interested.
EDIT:
I think I get it now ... A more general form of the posterior probability is
$\frac{r^{n+\alpha-1}(1-r)^{N-n+\beta-1}}{\int_0^1 r^{n+\alpha-1}(1-r)^{N-n+\beta-1} dr}$
But since they used an uninformative prior, $\alpha = \beta = 1$, it simplifies to
$\frac{r^{n}(1-r)^{N-n}}{\int_0^1 r^{n}(1-r)^{N-n} dr}$
Now the rest makes perfect sense as well. Thanks for getting me back on track.