# Posterior predictive distribution - dirichlet multinomial model

i saw this derivation of the posterior predictive distribution of a dirichlet multinomial model. Is one always allowed, or are there special circumstances, in which one can split the integral in compoenents and integrate seperately? I.e. can one always state that $p(\theta|D)$ can be split into $\int[\int p(\theta_-j,\theta_j|D)d\theta_-j]d\theta_j$ and subsequently be reduced to $\int p(\theta_j|D)d\theta_j$?

For any $\theta = (\theta_{1}, \ldots, \theta_{n})$ you always have the case that $\int p(\theta_{j}, \theta_{\neg j}) d\theta_{\neg j} = p(\theta_{j})$. You can call it the marginalization rule for joint densities.

The expression

$$\int p(X = j \, | \, \theta) p(\theta) d\theta$$

can be rewritten a bit more explicitly as the iterated integral

$$\int_{\theta_{j}} \int_{\theta_{\neg j}} p(X = j \, | \, \theta)p(\theta) d\theta_{\neg j} d\theta_{j}.$$

Since $p(X = j \, | \, \theta) = \theta_{j}$ in the single-trial multinomial case, that whole term doesn't depend on $\theta_{\neg j}$. You can just move that term out of the innermost integral, like so:

$$\int_{\theta_{j}}\theta_{j} \left( \int_{\theta_{\neg j}} p(\theta) d\theta_{\neg j}\right) d\theta_{j}.$$

Now you can just apply the 'marginalization rule' to evaluate the innermost integral (which I've put in parentheses for clarity), leaving you with

$$\int_{\theta_{j}}\theta_{j} p(\theta_{j})d\theta_{j}.$$

That Dirichlet distributions have marginal beta distributions lets you evaluate this last integral easily.