Mistake in derivation about categorical distribution and Dirichlet distribution?

Question

$p(x|\pi)$ follows the categorical distribution (the multinomial with one observation), where $\sum\pi_i=1$ and $x$ is a one-hot vector, and $p(\pi|\alpha)$ follows the Dirichlet distribution.

$p(x|\pi)=\prod\pi_i^{x_i}$

$p(\pi|\alpha)=\frac{1}{B(\alpha)}\prod\pi_i^{\alpha_i-1}$

$p(x,\pi|\alpha)=\frac{1}{B(\alpha)}\prod\pi_i^{\alpha_i-1+x_i}$

$p(x|\alpha)=\int p(x,\pi|\alpha)d\pi=\frac{1}{B(\alpha)}\int\prod\pi_i^{\alpha_i-1+x_i}d\pi=\frac{B(\alpha+x)}{B(\alpha)}$

$\sum p(x^{(i)}|\alpha)=\sum\frac{B(\alpha+x^{(i)})}{B(\alpha)}=\sum\frac{\alpha_i+1}{1+\sum\alpha_j}\neq1$

where $x^{(i)}$ is the one-hot vector with the i-th bit equal to 1.

please correct me where I'm wrong, thanks in advance.

Answer 1

I think the mistake was with simplifying the $B(\cdot)$ function. Suppose $x$ is a $K$ length vecto, and let $\sum_{k=1}^{K} \alpha_k = A$.

$$B(\alpha) = \dfrac{\prod_{k=1}^{K} \Gamma(\alpha_k)}{\Gamma(A )} $$

\begin{align*} \dfrac{B(\alpha+x)}{B(\alpha)} & = \dfrac{\prod_{k=1}^{K} \Gamma(\alpha_k + x_k)}{\Gamma(\sum_{k=1}^{K} \alpha_k + \sum_{k=1}^{K} x_k )}\dfrac{\Gamma(A )} {\prod_{k=1}^{K} \Gamma(\alpha_k)}\\ \end{align*}

Since only one of the components of $x$ can be 1, $\sum_{k=1}^{K} x_k = 1$. Also, note that $\Gamma(z + 1) = z\Gamma(z)$. Using this,

\begin{align*} \dfrac{B(\alpha+x)}{B(\alpha)} & = \dfrac{\prod_{k=1}^{K} \Gamma(\alpha_k + x_k)}{\Gamma(A + 1 )}\dfrac{\Gamma(A )} {\prod_{k=1}^{K} \Gamma(\alpha_k)}\\ & = \dfrac{\prod_{k=1}^{K} \Gamma(\alpha_k + x_k)}{\prod_{k=1}^{K} \Gamma(\alpha_k)}\dfrac{1}{A}\\ & = \dfrac{1}{A} \prod_{k=1}^{K}\dfrac{\Gamma(\alpha_k + x_k)}{\Gamma(\alpha_k)}\\ & = \prod_{k=1}^{K} \dfrac{1}{A} \dfrac{\Gamma(\alpha_k + x_k)}{\Gamma(\alpha_k)} = \prod_{k=1}^{K} p(x_k|\alpha). \end{align*}

To verify if this is a valid pdf \begin{align*} \sum_{k=1}^{K} p(x_k = 1|\alpha_k) & = \sum_{k=1}^{K}\dfrac{1}{A} \dfrac{\Gamma(\alpha_k + 1)}{\Gamma(\alpha_k)}\\ & = \sum_{k=1}^{K}\dfrac{\alpha_k}{A} =1. \end{align*}

I am not sure what steps you skipped, but I think you misused the $\Gamma(\cdot)$ function property.