i saw this derivation of the posterior predictive distribution of a dirichlet multinomial model. Is one always allowed, or are there special circumstances, in which one can split the integral in compoenents and integrate seperately? I.e. can one always state that $p(\theta|D)$ can be split into $\int[\int p(\theta_-j,\theta_j|D)d\theta_-j]d\theta_j$ and subsequently be reduced to $\int p(\theta_j|D)d\theta_j$?

enter image description here


1 Answer 1


For any $\theta = (\theta_{1}, \ldots, \theta_{n})$ you always have the case that $\int p(\theta_{j}, \theta_{\neg j}) d\theta_{\neg j} = p(\theta_{j})$. You can call it the marginalization rule for joint densities.

The expression

$$ \int p(X = j \, | \, \theta) p(\theta) d\theta $$

can be rewritten a bit more explicitly as the iterated integral

$$ \int_{\theta_{j}} \int_{\theta_{\neg j}} p(X = j \, | \, \theta)p(\theta) d\theta_{\neg j} d\theta_{j}. $$

Since $p(X = j \, | \, \theta) = \theta_{j}$ in the single-trial multinomial case, that whole term doesn't depend on $\theta_{\neg j}$. You can just move that term out of the innermost integral, like so:

$$ \int_{\theta_{j}}\theta_{j} \left( \int_{\theta_{\neg j}} p(\theta) d\theta_{\neg j}\right) d\theta_{j}. $$

Now you can just apply the 'marginalization rule' to evaluate the innermost integral (which I've put in parentheses for clarity), leaving you with

$$ \int_{\theta_{j}}\theta_{j} p(\theta_{j})d\theta_{j}. $$

That Dirichlet distributions have marginal beta distributions lets you evaluate this last integral easily.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.