I am a beginner in statistics, and am self-studying from "Information Theory, Inference, and Learning Algorithms" by David MacKay. I've hit a wall with one of the questions, and was wondering if any of you could be so kind as to point me in the right direction. Admittedly, my title may be wrong or totally misleading as well - if you found it was, please let me know what would have been a better title for this question!
The example which the questions are based on is as follows:
Bill tosses a bent coin N times, obtaining a sequence of heads and tails. We assume that hte coin has a probability fH of coming up heads; we do not know fH. If nH heads have occurred in N tosses, what is the probability distribution of fH? What is the probability that the N+1th outcome will be a head, given nH heads in N tosses.
The question is (this is Exercise 2.8 on p. 30):
Assuming a uniform prior on fH, P(fH) = 1, solve the problem in the example (given above). Sketch the posterior distribution of fH and compute the probability that the N+1th outcome will be a head for
- N = 3 and nH = 0;
- N = 3 and nH = 2;
- N = 10 and nH = 3;
- N = 300 and nH = 29;
He also states:
You will find the beta integral useful: (Since I'm new I guess I can only give a link to the image - sorry about that): http://i.stack.imgur.com/7MxQt.png
Now I'm guessing that my difficulty has a lot to do with the fact that I'm new to statistics, and still learning the parlance so to speak. So let me break my question into a few pieces:
- When he says "Sketch the posterior distribution of fH," what exactly does he mean by that? Does he mean it literally? (Yes...this is how ignorant I am...)
- Since he doesn't give us fH, is he expecting us to come up with it on our own? Would it be something like (fH + 1)/(N + 2)?
- First of all, how do I read the beta integral (what exactly is meant by Fa, Fb, pa) and secondly, how is it useful for this problem?
Like I said, I'm a beginner, and trying to grok as much of what he said, but I think at this point I need some help. Thanks in advance!