Moment generating function of multinomial distribution

Question

How would one find the moment generating function of the multinomial distribution, $\underline{X} \sim \mathrm{multinomial}(n, \underline{p})$? I know that by definition we have $$M_X (\underline{\theta}) = \mathbb{E} \exp{(\underline{\theta}^T X)} = \mathbb{E} \exp{\sum_{i=1}^k \theta_i X_i }$$ but then I can't see how we can go from here? I can't really get my head around what to do with the expected value of a vector. I don't think the $X_i$'s are independend so we can't factor out the expected value. Any explanation would be appreciated!

Answer 1

I will give the example with $k=2$ because it is more didactic, but you can generalize the solution. Before we start, let's remember that

$$ \sum_{x = 0}^n \frac{n!}{x!(n-x)!}a^xb^{n-x} = (a+b)^n. $$

By definition of the multinomial distribution we have

$$ P(X_1 = x_1, X_2 = x_2) = \frac{n!}{x_1!x_2!(n-x_1-x_2)!}p_1^{x_1}p_2^{x_2}(1-p_1-p_2)^{n-x_1-x_2}. $$

For now we fix $X_2 = x_2$, so we obtain

$$ \mathbb{E}\exp \theta_1 X_1 + \theta_2 x_2 = \\ \sum_{x_1=0}^{n-x_2}\frac{(n-x_2)!}{x_1!(n-x_1-x_2)!}\frac{n!}{x_2!(n-x_2!)}(p_1e^{\theta_1})^{x_1}(p_2e^{\theta_2})^{x_2}(1-p_1-p_2)^{n-x_2-x_1} =\\ \frac{n!}{x_2!(n-x_2!)}(p_1e^{\theta_1}+1-p_1-p_2)^{n-x_2}(p_2e^{\theta_2})^{x_2}. $$

We can now sum for all the values of $x_2$ between 0 and $n$ to obtain

$$ (p_1e^{\theta_1}+p_2e^{\theta_2}+1-p_1-p_2)^n, $$

which is the answer for $k=2$. The general result can be easily seen to be

$$ (p_1e^{\theta_1} + \ldots + p_ke^{\theta_k} + 1-p_1-\ldots-p_k)^n. $$