# Does the log likelihood become unimodal when the sample size goes to infinity?

I know that, under the usual regularity conditions, the MLE converges to the true parameter values as the sample size gets large. And the scaled MLE tends to being normally distributed. However, in a number of real world cases, finding the global maximum is difficult due to the presence of local minima in the likelihood. Intuitively, I would expect that the number and depth of the local minima would decrease as the sample size increases, leading to a unimodal likelihood. But is this true? Has it been proven?

• My intuition says no: I believe it is possible to construct a parametric distribution from which one can take infinitely many samples but there exist two distinct vector parameters for which the likelihood is maximized. – heropup Sep 9 '14 at 17:03
• "the MLE converges to the true parameter values" assumes that you have an algorithm that correctly locates the global maximum. It means that the sequence of values that achieve global maximas converges to the true parameter values, as sample size increases. – Khashaa Dec 25 '14 at 2:54
• @heropup That possibility is ruled out by the identification condition which is part and parcel of the regularity conditions. – Khashaa Dec 25 '14 at 3:04

For example, let $X_1,\dots,X_n$ be a random sample from the $k$-component mixture $$w_1\cdot\mathrm{N}(\mu_1,\sigma_1^2) + \dots + w_k\cdot\mathrm{N}(\mu_k,\sigma_k^2) \, ,$$ in which $w_i\geq 0$ and $\sum_{i=1}^k w_i=1$. Define $\theta_i=(w_i,\mu_i,\sigma_i^2)$, for $i=1\dots,k$, and $\theta=(\theta_1,\dots,\theta_k)$, and let $x=(x_1,\dots,x_n)$. The likelihood function is $$L_x(\theta) = \prod_{i=1}^n \sum_{j=1}^k w_j\cdot\frac{1}{\sqrt{2\pi}\sigma_j} e^{-(x_i-\mu_j)^2/2\sigma_j^2} \, .$$ Since for any permutation $\tau:\{1,\dots,k\}\xrightarrow{\rm 1:1}\{1,\dots,k\}$ we have $$L_x(\theta_1,\dots,\theta_k) = L_x(\theta_{\tau(1)},\dots,\theta_{\tau(k)}) \, ,$$ for this model the likelihood has at least $k!$ symmetric modes, no matter how large the sample size $n$ is.
• This seems a little beside the point because it's really reflecting the action of a symmetry group on the parameter space leading to an identifiability problem. That is, the space of distributions itself may have exactly one global extremum but the set of parametric representations of that extremum has $k!$ extrema merely because it has a $k!$-fold redundancy. Another simple example of this phenomenon would be estimating the center of a circular Normal distribution: the angle would be uncertain by an integral multiple of $2\pi$, giving infinitely many extrema. – whuber Sep 9 '14 at 17:47
• Your comment also seems to suggest that consideration of a suitable reparameterization of the model (with the introduction of special constraints, for example, to make $\text{parameter value}\mapsto\text{sampling distribution}$ injective, eliminating the "redudancy") would render void the nonidentifiability issue, but the literature in topics like (lack of) label switching in Gibbs sampling of mixture models indicates that this question is much harder. Sorry if I'm reading what you haven't written! – Zen Sep 9 '14 at 18:23