# What happens to the log likelihood when the maximum likelihood estimate does not exist?

What happens to the log likelihood (or indeed the likelihood) function, when the MLE does not exist?

The log likelihood is defined (for independent observations) as

$$l(\boldsymbol{\theta}) = \Sigma_{i=1}^N\text{ln}(P(y_i|\mathbf{x}_i,\boldsymbol{\theta}))$$

where the sum is over the observations, the $$y_i$$ are the endogenous variable values and the $$\mathbf{x_i}$$ are the values of the covariates in the ith observation.

The maximum likelihood estimator is

$$\boldsymbol{\theta}^* = argmax_\boldsymbol{\theta} \ l(\boldsymbol{\theta})$$

with a corresponding log likelihood

$$l(\boldsymbol{\theta}^*) = max_\boldsymbol{\theta} \ \Sigma_{i=1}^N\text{ln}(P(y_i|\mathbf{x}_i,\boldsymbol{\theta}))$$

I understand that this may depend on the model, and perhaps even on the covariate distribution. I am particularly interested in the logistic regression with i.i.d. Gaussian covariates, however more general answers or answers for other models/distributions would be most welcome.

• "Does not exist" in what sense? – Tim Aug 30 '19 at 18:36
• @Tim The data is completely or quasi separated. I believe then the maximum likelihood estimator is on the boundary of the domain (i.e. somewhere at infinity) see thiis paper by Albert and Anderson ,1984 jstor.org/stable/2336390?seq=1#metadata_info_tab_contents – Meep Aug 30 '19 at 18:49
• Have you reviewed our questions addressing perfect separation? What is it in particular that you would like to know? stats.stackexchange.com/… Is a sufficient answer to your question the observation that the likelihood for some parameter increases as you move towards $\pm \infty$? Why or why not? – Sycorax Aug 30 '19 at 19:03
• The asymptotics do not depend on the log likelihood or existence of MLE for any given set of data (essentially by definition): they depend on the model. Are you positing a model of complete separation? If so, the MLE is easy to work out from first principles and from that you can readily derive its asymptotic behavior. Obviously the log likelihood will almost surely not have a maximum for any dataset of any size in that case. – whuber Aug 30 '19 at 20:25