-1
$\begingroup$

Let's assume we have a discrete binary random variable K (K=0 or K=1) for which the prior distribution is binomial. My understanding of Bayesian statistics tells me that regardless of the likelihood, the posterior distribution for K can only be binomial as well. However, I have encountered a text which talks about "beta posterior for a binomial prior" -- am I correct that this is wrong?

$\endgroup$
  • $\begingroup$ Can you give more of the text? It probably means a Beta (hyper)prior for the binomial prior. And that Beta prior can be updated, so it has a prior value, a likelihood and a posterior. $\endgroup$ – Neil G May 8 '14 at 9:25
  • $\begingroup$ What is your observational data? You have prior distribution of $k\sim B(n,p)$. What is your observational data? And what is your conditional distribution? If your observational data is $k$, then it becomes the maximum likelihood estimation, with nothing to do with the posterior. $p(k|X)\propto p(k)L(k|X)$. What is your observational data $X$? If your observational data is your successful rate $p$, first of all, how do you get your observation of $p$? If it is, then what is $L(k|X)$? If $L(k|X)=\Pi f(X)$, then we have $p(k|X)=p(k)$, a binomial distribution. $\endgroup$ – Zhang Tschao May 8 '14 at 9:26
  • $\begingroup$ @NeilG No, the text doesn't discuss hyperpriors. $\endgroup$ – quant_dev May 8 '14 at 9:38
  • $\begingroup$ @quant_dev: They may not have used that term, but a Beta distribution is the conjugate prior for the Bernoulli distribution, which is the most likely explanation. Why don't you give more of the text? $\endgroup$ – Neil G May 8 '14 at 9:39
  • $\begingroup$ I cannot post the original text, buy it says about using multinomial logistic regression for modelling measured data. $\endgroup$ – quant_dev May 8 '14 at 9:56
1
$\begingroup$

1) Let $K\sim\text{Bernoulli}(p)$. When you observe a random sample, you get $n$ realizations of $K$ and try to guess $p$. So your random sample is a realization of the multiple random variable $(K_1,\dots,K_n)$.
$Y=\sum_{i=1}^n K_i$ is equal to the number of $K_i=1$, is the number of "successes" in your sample and $y/n$ ($y$ is the realization of $Y$) is your best guess about $p$. For example, if you get 100110101110, where $y=7$ and $n=12$, $p$ is likely close to $7/12$. So the binomial distribution of $Y$, $Y\sim\text{B}(n,p)$, is your likelihood, not your prior! You need a likelihood, not a prior, for $K$ -- better: for $y$. Then you need a prior for $p$ ($n$ is known).

2) The posterior always depends on both prior and likelihood, because is just their product. So you can never say that is something "regardless of the likelihood". Moreover, you are looking for a posterior for $p$ (the unknown parameter in $K\sim\text{Bernoulli}(p)$) conditional to the realizations of $K$, not a posterior for $K$.

3) If you assume that the prior for $p$ is $\text{Beta}(\alpha,\beta)$, then your posterior will be $\text{Beta}(y+\alpha,n-y+\beta)$, as you can see here. This is called "coniugacy" -- when prior and posterior belong to the same family -- and is computationally convenient; so this is why a beta prior is often used when the likelihood is binomial.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.