How would one find the moment generating function of the multinomial distribution, $\underline{X} \sim \mathrm{multinomial}(n, \underline{p})$? I know that by definition we have $$M_X (\underline{\theta}) = \mathbb{E} \exp{(\underline{\theta}^T X)} = \mathbb{E} \exp{\sum_{i=1}^k \theta_i X_i }$$ but then I can't see how we can go from here? I can't really get my head around what to do with the expected value of a vector. I don't think the $X_i$'s are independend so we can't factor out the expected value. Any explanation would be appreciated!


1 Answer 1


I will give the example with $k=2$ because it is more didactic, but you can generalize the solution. Before we start, let's remember that

$$ \sum_{x = 0}^n \frac{n!}{x!(n-x)!}a^xb^{n-x} = (a+b)^n. $$

By definition of the multinomial distribution we have

$$ P(X_1 = x_1, X_2 = x_2) = \frac{n!}{x_1!x_2!(n-x_1-x_2)!}p_1^{x_1}p_2^{x_2}(1-p_1-p_2)^{n-x_1-x_2}. $$

For now we fix $X_2 = x_2$, so we obtain

$$ \mathbb{E}\exp \theta_1 X_1 + \theta_2 x_2 = \\ \sum_{x_1=0}^{n-x_2}\frac{(n-x_2)!}{x_1!(n-x_1-x_2)!}\frac{n!}{x_2!(n-x_2!)}(p_1e^{\theta_1})^{x_1}(p_2e^{\theta_2})^{x_2}(1-p_1-p_2)^{n-x_2-x_1} =\\ \frac{n!}{x_2!(n-x_2!)}(p_1e^{\theta_1}+1-p_1-p_2)^{n-x_2}(p_2e^{\theta_2})^{x_2}. $$

We can now sum for all the values of $x_2$ between 0 and $n$ to obtain

$$ (p_1e^{\theta_1}+p_2e^{\theta_2}+1-p_1-p_2)^n, $$

which is the answer for $k=2$. The general result can be easily seen to be

$$ (p_1e^{\theta_1} + \ldots + p_ke^{\theta_k} + 1-p_1-\ldots-p_k)^n. $$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.