Why can I combine gamma and factorial coefficients for a beta conjugate posterior

Question

A commenter reminds me to be clearer.

In a Bayesian context, the product of a binomial likelihood and a beta prior probability is

$$\left( {\begin{array}{*{20}{c}}n\\x\end{array}} \right)p_{}^x{(1 - p)^{n - x}}\frac{{\Gamma (a + b)}}{{\Gamma (a)\Gamma (b)}}p_{}^{a - 1}{(1 - {p_{}})^{b - 1}} % MathType!MTEF!2!1!+- % feaagKart1ev2aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn % hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr % 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq-Jc9 % vqaqpepm0xbba9pwe9Q8fs0-yqaqpepae9pg0FirpepeKkFr0xfr-x % fr-xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaaeaaaaaaaaa8 % qadaqadaWdaeaafaqabeGabaaabaWdbiaad6gaa8aabaWdbiaadIha % aaaacaGLOaGaayzkaaGaamiCamaaDaaaleaaaeaacaWG4baaaOGaai % ikaiaaigdacqGHsislcaWGWbGaaiykamaaCaaaleqabaGaamOBaiab % gkHiTiaadIhaaaGcpaWaaSaaaeaacqqHtoWrcaGGOaGaamyyaiabgU % caRiaadkgacaGGPaaabaGaeu4KdCKaaiikaiaadggacaGGPaGaeu4K % dCKaaiikaiaadkgacaGGPaaaa8qacaWGWbWaa0baaSqaaaqaaiaadg % gacqGHsislcaaIXaaaaOGaaiikaiaaigdacqGHsislcaWGWbWaaSba % aSqaaaqabaGccaGGPaWaaWbaaSqabeaacaWGIbGaeyOeI0IaaGymaa % aaaaa!5A9D! $$

Ignoring the normalizing constant, and noting the correspondence between factorials and gamma values for integers, the posterior probability can be written as

$$\begin{array}{c}P\left( {\begin{array}{*{20}{r}}p\end{array}\left| {x,n,a,b} \right.} \right) = \frac{{\Gamma (n)}}{{\Gamma (x)\Gamma (n - x)}}p_{}^x{(1 - p)^{n - x}}\frac{{\Gamma (a + b)}}{{\Gamma (a)\Gamma (b)}}p_{}^{a - 1}{(1 - {p_{}})^{b - 1}}\\\\ = \frac{{\Gamma (n + a + b)}}{{\Gamma (x + a)\Gamma (b + n - x)}}p_{}^{x + a - 1}{(1 - p)^{n - x + b - 1}}\end{array} % MathType!MTEF!2!1!+- % feaagKart1ev2aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn % hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr % 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq-Jc9 % vqaqpepm0xbba9pwe9Q8fs0-yqaqpepae9pg0FirpepeKkFr0xfr-x % fr-xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGceaabbeaacaWGqb % WaaeWaaeaafaqaceqabaaabaGaamiCaaaadaabbaqaaiaadIhacaGG % SaGaamOBaiaacYcacaWGHbGaaiilaiaadkgaaiaawEa7aaGaayjkai % aawMcaaiabg2da9maalaaabaGaeu4KdCKaaiikaiaad6gacaGGPaaa % baGaeu4KdCKaaiikaiaadIhacaGGPaGaeu4KdCKaaiikaiaad6gacq % GHsislcaWG4bGaaiykaaaaqaaaaaaaaaWdbiaadchadaqhaaWcbaaa % baGaamiEaaaakiaacIcacaaIXaGaeyOeI0IaamiCaiaacMcadaahaa % Wcbeqaaiaad6gacqGHsislcaWG4baaaOWdamaalaaabaGaeu4KdCKa % aiikaiaadggacqGHRaWkcaWGIbGaaiykaaqaaiabfo5ahjaacIcaca % WGHbGaaiykaiabfo5ahjaacIcacaWGIbGaaiykaaaapeGaamiCamaa % DaaaleaaaeaacaWGHbGaeyOeI0IaaGymaaaakiaacIcacaaIXaGaey % OeI0IaamiCamaaBaaaleaaaeqaaOGaaiykamaaCaaaleqabaGaamOy % aiabgkHiTiaaigdaaaaakeaaaeaapaGaeyypa0ZaaSaaaeaacqqHto % WrcaGGOaGaamOBaiabgUcaRiaadggacqGHRaWkcaWGIbGaaiykaaqa % aiabfo5ahjaacIcacaWG4bGaey4kaSIaamyyaiaacMcacqqHtoWrca % GGOaGaamOyaiabgUcaRiaad6gacqGHsislcaWG4bGaaiykaaaapeGa % amiCamaaDaaaleaaaeaacaWG4bGaey4kaSIaamyyaiabgkHiTiaaig % daaaGccaGGOaGaaGymaiabgkHiTiaadchacaGGPaWaaWbaaSqabeaa % caWGUbGaeyOeI0IaamiEaiabgUcaRiaadkgacqGHsislcaaIXaaaaa % aaaa!9545! $$

what is the reason or better, derivation, for why the gamma coefficient in the second equation has that form? That is, why does one just add the parameter values, n, x, a, b ? One explanation I have seen is that one just recognizes the exponents in the second equation form a beta distribution, and simply forms the associated gamma coefficient with them. That is intuitive, but is there a derivation some where that shows why this is correct?

Answer 1

While the other answers are pretty much right, I think it's somewhat simpler to think about it like this. Bayes' theorem leads to the following:

$$ \require{cancel} \begin{array}{rcl} p(\theta|D) &=& \dfrac{p(D|\theta)p(\theta)}{\int_{\Theta}p(D|\theta)p(\theta)d\theta}\\[2ex] &=& \dfrac{\binom{n}{x}\theta^x(1-\theta)^{n-x}Be(\alpha,\beta)\theta^{\alpha-1}(1-\theta)^{\beta-1}}{\int_\Theta\binom{n}{x}\theta^x(1-\theta)^{n-x}Be(\alpha,\beta)\theta^{\alpha-1}(1-\theta)^{\beta-1}d\theta} \end{array} $$ Since neither the Binomial nor Beta function depend on theta, they can be moved outside the integral in the denominator, so they cancel out and you're left with the typical normalizing Beta function. $$ \require{cancel} \begin{array}{rcl} p(\theta|D) &=& \dfrac{\binom{n}{x}Be(\alpha,\beta)\theta^{x+\alpha-1}(1-\theta)^{n-x+\beta-1}}{\binom{n}{x}Be(\alpha,\beta)\int_\Theta\theta^{x+\alpha-1}(1-\theta)^{n-x+\beta-1}d\theta}\\[2ex] &=&\dfrac{\cancel{\binom{n}{x}}\cancel{Be(\alpha,\beta)}\theta^{x+\alpha-1}(1-\theta)^{n-x+\beta-1}}{\cancel{\binom{n}{x}}\cancel{Be(\alpha,\beta)}\int_\Theta\theta^{x+\alpha-1}(1-\theta)^{n-x+\beta-1}d\theta}\\[2ex] &=&\dfrac{\theta^{x+\alpha-1}(1-\theta)^{n-x+\beta-1}}{Be(\alpha+x,\beta+n-x)} \end{array} $$

(Incidentally this is why the posterior is unaffected by stopping rules, since as long as the stopping rule doesn't affect the kernel the terms can move outside the integral and cancel)

Answer 2

I think what you are missing here is the understanding of proportionality. Let $f(x\mid p)$ be the Binomial likelihood and $f(p)$ be the Beta$(a,b)$ prior. Then the posterior $$f(p\mid x) = \dfrac{f(x\mid p)f(p)}{f(x)} \propto f(x \mid p)f(p). $$

Here $f(x)$ is a constant for $p \mid x$, and thus we can ignore it. So when calculating the posterior (without ignoring constants for now), $$f(p \mid x) = \dfrac{1}{f(x)} \dfrac{\Gamma(n) \Gamma(a+b)}{\Gamma(x)\Gamma(n-x)\Gamma(a)\Gamma(b)}p^{x+a-1}(1-p)^{n-x+b-1}.$$

Then using the method mentioned by @RaxYao, you figure out that this is a Beta$(x+a, n-x+b)$ which means that, the constants will equate to the constant for the Beta$(x+a, n-x+b)$. That is,

$$\dfrac{1}{f(x)} \dfrac{\Gamma(n) \Gamma(a+b)}{\Gamma(x)\Gamma(n-x)\Gamma(a)\Gamma(b)} = \dfrac{\Gamma(n+a+b)}{\Gamma(x+a)\Gamma(n-x+b)} $$

So it is not that $\Gamma(x)\Gamma(a) = \Gamma(x+a)$, but in fact,

$$f(x) = \dfrac{\Gamma(x+a)\Gamma(n-x+b) \Gamma(n) \Gamma(a+b)}{\Gamma(n+a+b)\Gamma(x)\Gamma(n-x)\Gamma(a)\Gamma(b)}.$$

Answer 3

The reason is pretty simple. We can compute the coefficient in the following way.

In the Bayesian point of view, the posterior of $p$ is:

$$\begin{align} P\left( {p|{x,n,a,b}} \right) & \propto p^x{(1 - p)^{n - x}}p^{a - 1}{(1 - {p_{}})^{b - 1}} \\ & \propto p_{}^{x + a - 1}{(1 - p)^{n - x + b - 1}} \end{align} $$ Notice that the to make the posterior of $p$ of the form $p_{}^{x + a - 1}{(1 - p)^{n - x + b - 1}} $ and make it to be a proper PDF, i.e., $$ \begin{align} \int P\left( {p|{x,n,a,b}} \right) dp & = 1 \\ \implies \int_p c \times p_{}^{x + a - 1}{(1 - p)^{n - x + b - 1}} & = 1 \end{align} $$ Then it will be very simple to determine the value of $c$. Notice that the Beta distribution with parameter $(x+a, n-x+b)$ can make the last equation feasible. Thus $$c= \frac{\Gamma(x+a+n-x+b)}{\Gamma(x+a)\Gamma(n-x+b)}=\frac{\Gamma(a+n+b)}{\Gamma(x+a)\Gamma(n-x+b)} $$

We say that $p_{}^{x + a - 1}{(1 - p)^{n - x + b - 1}}$ is the kernel of a distribution, which happens to follow the pattern of a Beta distribution.