Find the Fisher information matrix for the MLE of a Multinomial Please help me how to find the Fisher information matrix for the MLE of a Multinomial ...
It is said that considering $n$ and $p$ constraints in multinomial distribution.
In other words, when $p$ has $k$-dimension, real parameter number is $k-1$.
 A: The Fisher information function is the variance of the score function, so you start by finding the latter.  If you have an observed data vector $\mathbf{X} \sim \text{Mu}(\mathbf{p})$ using the probability vector $\mathbf{p} =(p_1,...,p_k)$ then you get the log-likelihood function:
$$\ell_\mathbf{x}(\mathbf{p}) = \text{const} + \sum_{i=1}^k x_i \log(p_i),$$
which gives you the score function:
$$s_\mathbf{x}(\mathbf{p}) \equiv \nabla \ell_\mathbf{x}(\mathbf{p}) = \bigg( \frac{x_1}{p_1},...,\frac{x_k}{p_k} \bigg) = \text{diag}(1/\mathbf{p}) \mathbf{x}.$$
Consequently, the Fisher information matrix is:
$$\begin{align}
\mathcal{I}(\mathbf{p})
\equiv \mathbb{V}(s_\mathbf{X}(\mathbf{p})) 
&= \mathbb{V}(\text{diag}(1/\mathbf{p}) \mathbf{X}) \\[12pt]
&= \text{diag}(1/\mathbf{p}) \mathbb{V}(\mathbf{X}) \text{diag}(1/\mathbf{p}) \\[12pt]
&= n \text{diag}(1/\mathbf{p}) [\text{diag}(\mathbf{p}) - \mathbf{p}\mathbf{p}^\text{T}] \text{diag}(1/\mathbf{p}) \\[12pt]
&= n [\text{diag}(1/\mathbf{p}) - (\text{diag}(1/\mathbf{p}) \mathbf{p}) (\text{diag}(1/\mathbf{p}) \mathbf{p})^\text{T}] \\[12pt]
&= n [\text{diag}(1/\mathbf{p}) - \mathbf{1} \mathbf{1}^\text{T}] \\[12pt]
&= n \begin{bmatrix}
\frac{1-p_1}{p_1} & -1 & -1 & \cdots & -1 \\
-1 & \frac{1-p_2}{p_2} & -1 & \cdots & -1 \\
-1 & -1 & \frac{1-p_3}{p_3} & \cdots & -1 \\
\vdots & \vdots & \vdots & \ddots & \vdots \\
-1 & -1 & -1 & \cdots & \frac{1-p_k}{p_k} \\
\end{bmatrix}.
\end{align}$$
The above treatment is for the full parameter vector with $k$ elements, but you can reduce this to the corresponding parameter vector with $k-1$ elements by taking $p_k = 1-\sum_{i=1}^{k-1} p_i$.  In the latter case you would rewrite the score function using this equivalence and then you would get a $(k-1) \times (k-1)$ invertible variance matrix.  I will leave this as a useful exercise for you.
