MLE Asymptotic Normality regularity conditions

Question

I had this lecture of mathematical statistics about asymptotic normality of MLE. In order to prove this, a series of regularity conditions were stated, and the identifiability condition was among them.

Given a random sample $X=(X_1,...,X_n)$, the identifiability condition was stated like this: $$\mathbb{E}_{\theta_1}[S(\theta_2,X)]=0\iff\theta_1=\theta_2$$where $S(\theta,X)$ is the score function, i.e. $S(\theta,X)=\frac{d}{d\theta}\log L(\theta|X)$, where $L(\theta|X)$ is the likelihood function. However, as far as I know, identifiability condition generally states that: $$L(\theta_1|X)=L(\theta_2|X)\iff\theta_1=\theta_2$$

So, my first question is if there is some kind of relationship or equivalence between these conditions, or if there is any reference to search more about this. And my second question refers to another regularity condition used in the proof, that is the following:

$$\sup_{\theta_2\in\Theta}\left|M(\theta,\theta_2)-\left|\frac{S(\theta_2,X)}{n}\right|\right|=0, M(\theta,\theta_2)=\frac{1}{n}\left|\mathbb{E}_{\theta}[S(\theta_2,X)]\right|$$

Regarding this condition, I think I didn't get some kind of intuition of what it really means, I only know that it is necessary for this particular version of the proof. If someone would be kind to give some reference on this topic or clarify these questions, I would be very grateful.

Answer 1

The score condition here is not quite the same as identifiability of the parameters, but it's close. Suppose that $\log L$ is a smooth function of $\theta$ and the maximum doesn't occur at a boundary, so that the MLE must solve the score equation, and the true parameter must solve the expected score equation $E_{\theta_{true}}[S(\theta_{true}]=0$.

The condition says that the score equation will not (in expectation) have any zero other than the true parameter value. If the model were not identifiable in the likelihood sense, then any parameter value that can't be distinguished from the true parameter will also solve the score equation, and we will have $$E_{\theta_{true}}[S(\theta_{\textrm{other}}]=0$$.

But the score condition is stronger than identifiability of the parameter: it also rules out solutions to the expected score equation that are just local maxima and minima of the expected loglikelihood. It's also stronger in requiring that the loglikelihood is differentiable and that the MLE doesn't occur at a boundary of the parameter space.

You can get by with much weaker conditions, such as that the loglikelihood is bounded away from its maximum value for $\theta$ not in a neighbourhood of the maximum.

Your second condition is also strong. It says that the observed value of the score is close to the expected value uniformly over $\Theta$. Since the expected score equation has its only zero at $\theta_{\mathrm{true}}$, the score function from the data can only be close to zero at $\theta$ near $\theta_{\mathrm{true}}$, and so the solution of the score equation from the data must be increasingly close to $\theta_{\mathrm{true}}$.

You don't say what $\Theta$ is. Having this hold for the entire parameter space is a very strong condition. Having it hold for a compact neighbourhood of $\theta_{\mathrm{true}}$ is a much more reasonable condition. Again, though, it's sufficient to have a condition like that for the log likelihood, rather than the score, which lets you deal with models like the Laplace distribution $f(x;\theta)=\exp(-|x-\theta|)$ whose loglikelihood is not everywhere differentiable.