For MLE, why does the information inequality imply identifiability

Question

Let $X = \langle X_1, \dots, X_n \rangle^{\top}$ be a finite sample of observation $X$ where $X \sim \mathbb{P}_{\theta_0}$ with $\theta_0 \in \Theta$ and density $f_X(x; \theta_0)$. The true parameter $\theta_0$ is globally identifiable if

$$ \forall \theta \neq \theta_0 \implies \mathbb{P}[f_X(x; \theta) \neq f_X(x; \theta_0)] > 0$$

My lecture notes then say that because of the information inequality for the log likelihood,

$$ \mathbb{E}_{\theta_0} [\ell_n(\theta)] \leq \mathbb{E}_{\theta_0}[\ell_{n}(\theta_0)], \qquad \forall \theta \in \Theta \subseteq \mathbb{R}^d. $$

this implies global identification, i.e.

$$ \mathbb{E}_{\theta_0} [\ell_n(\theta)] < \mathbb{E}_{\theta_0}[\ell_{n}(\theta_0)], \qquad \forall \theta \neq \theta_0. $$

I don't follow this last step. Why does the inequality become strict? How is this last inequality related to the definition of global identification?

Answer 1

Let $W$ and $U$ denote two continuous (for simplicity) r.v.s with densities $f_W(w)$ and $f_U(u)$. The expected value of the r.v. $Z:=f_W(U)/f_U(U)$ is \begin{eqnarray*} E(Z)&=&\int_{S_U}\frac{f_W(u)}{f_U(u)}rd F_U(u)\\ &=&\int_{S_U}f_W(u)d u\\ &\leqslant&1, \end{eqnarray*} where $S_U$ is the support of $f_U(u)$. Then, we use Jensen's inequality to get $$E[\log(f_W(U)/f_U(U))]\leqslant\log(E[f_W(U)/f_U(U)])\leqslant\log(1)=0$$ Now, take $\log f_W(U)=L(\theta;U)$ and $\log f_U(U)=L(\theta_0;U)$, so that $$ E[L(\theta;U)]-E[L(\theta_0;U)]=E[\log(f_W(U)/f_U(U))]\leqslant 0, $$ which establishes the weak inequality.

Now, under global identification, $Z$ is a nondegenerate r.v., so that the inequality is strict.