I'm taking a 3rd year statistical theory class in which we're introducing the likelihood function with a bit more mathematical rigor.

Under the discussion for maximizing the likelihood function $\mathcal{L}(\theta)$, the class notes mention that if it were the case that the parameter space $\Theta$ is an open set (every point is an interior point), then we can maximize the log likelihood function which is very nice in practice.

However, in the case that the parameter space $\Theta$ was a closed set, then it is better to maximize the likelihood function directly.

I'm just wondering why is that the case? Assuming that the MLE $\hat\theta$ is a boundary point of $\Theta$, why does this matter for using the log-likelihood instead of the likelihood function directly.

Also, as an aside, I tried looking for answers online before bringing this question here. The only relevant information was relating convex parameter spaces and strictly concave likelihood functions as well as uniqueness of solutions. Is this relevant to my question above? Appreciate any help I can get.

  • 3
    $\begingroup$ I don't follow your premises: there's no assurance the likelihood is maximized in an open parameter space. Indeed, precisely the opposite is usually true: a continuous function (which is usually the case for likelihoods) will have a maximum on a closed, compact set, but needn't have one on an open set. It's also unclear in what sense you use "better:" does that refer to numerical stability? If not, then (since mathematically maximizing the likelihood and maximizing its log are identical problems) what does it mean?? $\endgroup$ – whuber Feb 21 at 0:14
  • $\begingroup$ I apologize if I was unclear. The exact text from the slides is as follows: Two general scenarios for finding $\hat{\theta}$ that maximizes the likelihood: 1) $\mathcal{L}(\theta)$ is differentiable and the parameter space $\Theta$ is an open set (i.e. every point of $\Theta$ is an interior point), then $\hat{\theta}$ (if it exists) satisfies the likelihood equation $$\frac{d}{d\theta} ln \mathcal{L}(\hat{\theta}) = 0 $$. 2) $\hat{\theta}$ occurs at a boundary (i.e. $\Theta$ is not an open set). In these cases, we need to directly maximize $\mathcal{L}(\theta)$ $\endgroup$ – Amin Sammara Feb 21 at 0:56
  • $\begingroup$ My question is why do we make a distinction whether $\Theta$ is an open set or not. Why does this matter when it comes to maximizing the log likelihood. $\endgroup$ – Amin Sammara Feb 21 at 1:02
  • 1
    $\begingroup$ See en.wikipedia.org/wiki/Extreme_value_theorem. $\endgroup$ – whuber Feb 21 at 13:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.