3
$\begingroup$

Suppose that for $\theta ∈ (0, 1)$, $X$ is a continuous random variable with density $f_\theta(x) = \frac{3(1 − \theta)} {4\delta^3(\theta)} [\delta^2(\theta) − (x − \theta)^2]\mathbb{1}(|x − \theta| ≤ \delta(\theta)) + \frac{\theta}{2} \mathbb{1}(|x| ≤ 1)$ where $\delta(\theta) > 0$ for all $\theta$.

I'm struggling to show that the maximum likelihood estimator exists. Computing $f'$ and $f''$ gives nothing. Is there any hint? Thanks in advance.

$\endgroup$
8
  • 2
    $\begingroup$ For me a first step would be to try to draw the likelihood. Often that provides a helpful clue to progress. $\endgroup$
    – Glen_b
    Commented Feb 4, 2023 at 2:20
  • $\begingroup$ Not possible in this case. We don't have info about $\delta$ function. $\endgroup$
    – user115608
    Commented Feb 4, 2023 at 2:23
  • 2
    $\begingroup$ Can you double-check the source of this problem to make sure no additional information is missing? I am asking because there is a very similar example appeared in A Course in Large Sample Theory (pp. 116 -- 117) by Thomas Ferguson, where I saw additional information about the domain of $x$ and properties of $\delta(\theta)$. $\endgroup$
    – Zhanxiong
    Commented Feb 4, 2023 at 5:44
  • 2
    $\begingroup$ For the score function to exist, $\delta(\cdot)$ must be a.e. differentiable. $\endgroup$
    – Xi'an
    Commented Feb 4, 2023 at 9:30
  • 2
    $\begingroup$ Even when you can't pin down a function like $\delta$ specifically, pick one that satisfies your condition and make a plot as @Glen_b recommends: that's almost always helpful. It's curious that you object you "don't have info" while in your question you refer to "computing" the first two derivatives of $f,$ which is possible only when you have lots of information about $\delta$! $\endgroup$
    – whuber
    Commented Feb 4, 2023 at 15:49

1 Answer 1

6
$\begingroup$

The properties of the function $\delta(\theta)$ are important for the existence of a maximum likelihood estimate.

Cases where the MLE is not unique

Example: let the observation be $x = 0$, then the likelihood function is

$$\mathcal{L}(\theta|x=0) = \theta \left(\frac{1}{2} \right)+ \left(1-\theta\right) \times \frac{3}{4\delta(\theta)} \left( 1 - \left(\frac{\theta}{\delta(\theta)}\right)^2\right)$$

Then for some suitably chosen $\delta(\theta)$ you can have a maximum in both $\theta = 1$, with value $0.5$, as well as in the point $\theta = 0$ with value $0.5$ as well if $\delta(0) = 3/2 $.

In the extreme case, possibly you could find a function $\delta(\theta)$ that makes the likelihood function constant in the entire range. That would require you to solve for a function $\delta(\theta)$ that makes the likelihood constant. Possibly one can solve the related differential equation, but the idea that the function can have maximum's in two points is already sufficient.

Cases where the likelihood is unbounded, or bounded but without maximum

Your interval is not a closed interval. So you may have a situation where the likelihood function increases for $\theta \to 0$ or $\theta \to 1$, and a single point where the lowest upper bound is obtained does not exist.

Example: let the observation be $x = 0$, and $\delta(\theta) = 2\theta$ then

$$\mathcal{L}(\theta|x=0) = \theta \left(\frac{1}{2} \right)+ \left(1-\theta\right) \times \frac{9}{32\theta}$$

example

This situation, where the upper bound can be infinite can occur when $\delta(\theta)$ can approach $0$ (without actually being equal to $0$).

The situation, where the lowest upper bound is finite, but can still not be obtained can occur when the function is discontinuous or the interval is not closed. (the sufficient conditions of the extreme value theorem)

$\endgroup$
7
  • $\begingroup$ In my opinion, having two maximums is not sufficient to conclude that MLE does not exist (it just means MLE may not be unique). I am inclined that given very mild conditions on $\delta$ (say, it is continuous), then MLE indeed always exists. $\endgroup$
    – Zhanxiong
    Commented Feb 4, 2023 at 21:59
  • $\begingroup$ @Zhanxiong , fair enough, then the MLE exists, whenever a (not necessarily unique) global maximum exists. I am not sure whether continuity of $\delta$ is relevant. A different problem is that the set is on an open interval. If $\delta(\theta) = \theta$ and $x = 0$ then we get $\mathcal{L}(\theta) \to \infty$ for $\theta \to 0$. $\endgroup$ Commented Feb 4, 2023 at 22:21
  • $\begingroup$ Ah, continuity is relevant as well. We can have some $\delta(\theta)$ that approaches zero for $\theta$ approaching some value but $\delta(\theta)$ is not zero when $\theta$ is equal to that value. $\endgroup$ Commented Feb 4, 2023 at 22:24
  • $\begingroup$ I actually had the same idea (to construct $\delta(\theta)$ so that $L(\theta)$ becomes unbounded on $(0, 1)$). However, after careful calculation, $\delta(\theta) = \theta$ and $x = 0$ actually gives $L(\theta) = \frac{\theta}{2}$ (of course, if $\Theta$ is open, then it is still fair to say MLE does not exist). $\endgroup$
    – Zhanxiong
    Commented Feb 4, 2023 at 22:27
  • $\begingroup$ This density is in fact a classical example (similar density appeared in both Ferguson's and Lehmann's book) to show that MLE exists, but not necessarily consistent (of course, in these references, $\delta(\theta)$ is assumed to be continuous decreasing with $\delta(0) = 1$ and $0 < \delta(\theta) \leq 1 - \theta$). $\endgroup$
    – Zhanxiong
    Commented Feb 4, 2023 at 22:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.