need help understanding Dirichlet (coursera's PGM class week 7 - Bayesian prediction) I'm trying to work through Coursera's probabilistic graphical models class (week 7: Baeysian prediction) and a have several questions. 


*

*In the Dirichlet distribution, I'm having difficulty trying to understand why there's a   -1    in theta's exponent:
$$P(\theta)=Dir(\alpha_1, ..., \alpha_k) = \frac{1}{Z} \cdot \prod_{j} \theta_{j}^{\alpha_{j}-1}$$

*How do you get from here:
$$P(X)=\int_{\theta}P(X|\theta)P(\theta)d\theta$$
to here:
$$P(X=x^{i}|\theta) = \int_{\theta} \frac{1}{Z} \cdot \theta_{i} \prod_{j} \theta_{j}^{\alpha_{j}-1}$$

*Also, how do you step through the integration for the following?:
$$\int_{\theta} \frac{1}{Z} \cdot \theta_{i} \prod_{j} \theta_{j}^{\alpha_{j}-1} = { \alpha_{i}\over{\sum_{j} \alpha_{j}} }$$


These are the lecture notes. My questions refer to the first slide.
 A: Just thought I'd add an example of how to calculate the 
normalising constant.
If you know the beta integral, then its easier to use that for direct integration.  With a change of variables in the usual definition you get
$$\int_{L}^{U}(x-L)^{a-1}(U-x)^{b-1}dx=(U-L)^{a+b-1}B(a,b)$$
The change in variables is $t=\frac{x-L}{U-L}$ and you get back to the standard definition of the beta integral.  To apply this to the calculation of Z we must first determine the limits of integration.  This is simple for the simplex as the parameters must all be positive and sum to 1.  So we have
 $$0\leq\theta_1\leq 1$$ 
$$0\leq\theta_i\leq 1-\sum_{j=1}^{i-1}\theta_j\;\;\; i=2,\dots,n-1$$
$$\theta_n=1-\sum_{j=1}^{n-1}\theta_j $$
This assumes that we integrate in the order $\theta_n,\theta_{n-1},\dots,\theta_1$.  The order of integration doesn't matter, but this order is easier to write down.The first integral is a substitution so we have for the second integral.
$$\int_{0}^{1-\sum_{j=1}^{n-2}\theta_j}\left[\prod_{k=1}^{n-2}\theta_{k}^{\alpha_k-1}\right]\theta_{n-1}^{\alpha_{n-1}-1}\left( 1-\sum_{j=1}^{n-2}\theta_j - \theta_{n-1}\right)^{\alpha_n-1}d\theta_{n-1}$$
This is of the form of the transform beta integral with $L=0$ and $U= 1-\sum_{j=1}^{n-2}\theta_j $  hence we get:
 $$\left[\prod_{k=1}^{n-2}\theta_{k}^{\alpha_k-1}\right]B(\alpha_n,\alpha_{n-1})\left( 1-\sum_{j=1}^{n-2}\theta_j \right)^{\alpha_n+\alpha_{n-1}-1}$$
Now we apply this again to the integral over $\theta_{n-2}$.  It is another transformed beta integral but with $U= 1-\sum_{j=1}^{n-3}\theta_j$.  Hence we get
$$\left[\prod_{k=1}^{n-3}\theta_{k}^{\alpha_k-1}\right]B(\alpha_n,\alpha_{n-1}) B(\alpha_n+\alpha_{n-1},\alpha_{n-2}) \left( 1-\sum_{j=1}^{n-3}\theta_j \right)^{\alpha_n+\alpha_{n-1}+\alpha_{n-2}-1}$$
It is now straight forward to repeatedly apply this and you get
$$Z= B(\alpha_n,\alpha_{n-1}) B(\alpha_n+\alpha_{n-1},\alpha_{n-2}) B(\alpha_n +\alpha_{n-1}+\alpha_{n-2} ,\alpha_{n-3}) \dots B(\alpha_n+\dots+\alpha_{2},\alpha_1)$$
If you plug in the relation between the beta and gamma integrals $B(a,b)=\frac{\Gamma(a)\Gamma(b)}{\Gamma(a+b)}$ you get the correct normalising constant.
A: In (2), it should be $P(X = x_i)$, not $P(X = x_i | \theta)$; the equation is arrived at by iterated expectation, i.e. $P(X = x_i) = E[P(X = x_i | \theta)] = E\theta_i = \int \frac 1 Z \theta_i \left(\prod \theta_j ^{\alpha_j - 1}\right) \ d\vec\theta$. (3) is easy if you take the form of the normalizing constant as known; it is $$
\frac 1 {Z(\alpha_1, ..., \alpha_n)} = \frac{\Gamma(\sum \alpha_j)}{\prod \Gamma(\alpha_j)}.
$$
The form of the normalizing constant can be found (e.g.) by exploiting the fact that the Dirichlet arises from norming a collection of gamma random variables with shape parameters given by the $\alpha_i$ by their sum. So the integral is $$
\frac{1}{Z(\alpha_1, ..., \alpha_n)} \int \theta_i ^{\alpha_i + 1 - 1} \prod_{i \ne j} \theta_j ^{\alpha_j - 1} \ d\vec\theta = \frac{Z(\alpha_1, ..., \alpha_i + 1, ..., \alpha_n)}{Z(\alpha_1, ..., \alpha_n)}
$$
since the integrand is the kernel of a Dirichlet with parameters $(\alpha_1, ..., \alpha_i + 1, ..., \alpha_n)$. Plugging in the form of $Z(\cdot)$ and using the properties of the gamma distribution leads to the result in (3). 
In all the integrals it is understood that $\theta_n := 1 - \sum_1 ^ {n - 1} \theta_i$, and the integrals are over the $n - 1$ dimensional simplex. 
