2
$\begingroup$

Assuming that I performed n iid tests, and the total number of test is n which is a fixed value, and the observaton of 1 which corresponding to successful results is X observations yeild with probability p, I know X a random variable draw from binomial distribution B(n,p). Now, assuming the prior P(p) from an uniform distribution, I want to calculate a posterior probability P(p|n,X) with respect to different X values.

From some textbooks, I know the conjugate prior for the binomial distritbution is a beta distribution. But the deta distribution is for continueous random variables. So how can I calculate the posterior P(p|n,X)?

Shall I use beta-binomial distribution instead of beta distribution to calculate the posterior probabilty P(p|n,X)?

I tried to calculate the posterior P(p|n,X) directly, using Bayesian rule, but I'm not sure wehther or not I can calculate P(p|n,X) in this way. Suppose n is fixed, P(p) is uniform distribution. My method is that I simply calculate the posterior P(p|n,X) from P(X|n,p)~B(n,p) with different p values. For instance, I calculated a 100X100 2D table of P(X|n,p) with different X (X as the first dimension of the table, range form 1 to n, with interval 1) and p (p as the second dimension of the table range form 0.01 to 1.0, with interval 0.01), then the posterior probability P(p|n,X) for each discretized p with respect to praticular X value from the 2D table by using Bayesian rule. Is this a correct way to calculate the P(p|n,X)?

|cite|improve this question
$\endgroup$
  • 1
    $\begingroup$ Your title seems to suggest a misconception. A posterior is for a parameter, not for a distribution. So you're talking about the posterior distribution of the parameter $p$ in a binomial. Note that $p$ is a continuous quantity -- it can take any value in (0,1); reasonable priors will generally be continuous distributions on that interval, and in that case posteriors will also be continuous distributions on the same interval. (Please try to fix the spelling and grammar errors in your post.) $\endgroup$ – Glen_b -Reinstate Monica Jan 29 '16 at 12:11
  • $\begingroup$ Thanks, yes, they are in the same interval. Let me try to correct the errors. $\endgroup$ – J.Doe Jan 29 '16 at 13:59
  • $\begingroup$ thanks for the comment, I changed the title for the post $\endgroup$ – J.Doe Jan 29 '16 at 14:14
3
$\begingroup$

A uniform distribution is a special kind kind of beta distribution, which for the binomial distribution we know that it is a conjugate distribution. This means that if we combine the binomial likelihood with our beta prior, then our posterior will be a beta distribution.

Note that assuming a beta prior corresponds to considering the parameter $p$ as a continuous random variable on the (0,1) interval. This needs not be the case in general. For instance, we may assume that the prior is just a two-point distribution, for instance $P(\theta = 0.3) = 0.7$ and $P(\theta = 0.7) = 0.3$, and our posterior will also be a two-point distribution.

Returning to the uniform prior case, we know that our posterior will be of the form

$$p\left(\theta|\mathbf{x} \right) \propto f\left(\mathbf{x} |\theta \right) g\left(\theta\right) = \binom{n}{x} \theta^x \left(1-\theta\right)^{n-x} 1, \quad 0<\theta<1 $$

And you can determine the normalizing constant by figuring out what you need for this integrate to one (as a function of $\theta$). Once you plug in the values of $n$ and $x$ you can plot this as a function of $\theta$ and determine the value you like the most, e.g. mean, median, mode etc. Where did you get stuck?

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Thanks John, your comments solved most my concern. Another question is about the normalize constant, i think the normalize constant calculate by integration to 1 can only be applied to calculate P(p|n,x) when P(p|n,x) is a continuous function, for the discretize senario, intergration to 1 might not be work, how do you think about the method that I mentionn above by using table to calculate the posterior P(p|n,x), i think using this table we even don't need to dervie the equation for posterior, we can directly calculate P(p|n,x) for each x and p with fixed n. $\endgroup$ – J.Doe Jan 29 '16 at 13:56
  • $\begingroup$ I tried to calculate some examples, for instance, when n=10, theta = 0.5, x = 5, the P(p|n,X) is a number much greater than 1 when using the normalize constant for continuous function. $\endgroup$ – J.Doe Jan 29 '16 at 14:05
  • $\begingroup$ @J.Doe The normalization factor depends on the prior distribution of course. If it is a two point distribution, as in my example, you don't integrate. You apply the theorem of Bayes and sum over all possible values. $\endgroup$ – JohnK Jan 29 '16 at 21:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.