3
$\begingroup$

I have a conceptual doubt. For example, suppose I have $X_i \stackrel{iid}{\sim} N(\theta^*,1)$ and I know that (I have the information) $\theta^{*}\geq 0$. So I have the Constrained Maximum Likelihood Estimation:

$$\hat{\theta}_{CMLE} = \max \{\bar{X}_n ,0 \}$$

I know, that if $\theta > 0$, then

$$\sqrt{n}(\hat{\theta}_{CMLE} - \theta^{*}) \to^{d} N(0,1)$$

I have heard that if $\theta^{*}$ is very close to zero, then I will not have a good approximation. For example, if $\theta^{*}= 0.00000000000000001$. Why should not I have a good approximation? If I know that although this $\theta^{*}$ is very small it will always be positive and my convergence above will be valid. In what sense would this not be a case of good approximation?

$\endgroup$
5
  • 2
    $\begingroup$ It's a good topic for questions. Please note, however, that the sense in which the left hand side converges (in distribution) to a standard Normal law is that the estimator $\hat\theta$ is the random variable, not $\theta^{*}.$ Because you truncate the sample mean at zero, what does that do to the distribution of the estimator? If you're not sure, run a quick simulation. The insight this easy exercise provides will carry over to much more complex situations. $\endgroup$
    – whuber
    Commented Mar 16, 2019 at 15:00
  • $\begingroup$ If I did not have the restriction, then the estimator would be $\bar{X}_n$. And this estimator has a distribution in finite samples given by $N (0,1)$. Let me see if I understand your question. Do you want to know the distribution of the constrained estimator? I can deduce what the distribution of $\sqrt{n}(\hat{\theta}_{CMLE} - \theta^{*})$ will be, independent of the true parameter will be zer or not. Is that what you want? $\endgroup$
    – Fam
    Commented Mar 16, 2019 at 15:12
  • 1
    $\begingroup$ Be a little careful. The distribution of ${\bar X}_n$ is not standard Normal: it is Normal with mean $\theta^{*}$ and variance $1/n.$ Thus, the definition of $\hat\theta$ censors this distribution by replacing all negative values with $0.$ It can help to draw the PDF of ${\bar X}_n$ and to sketch the area corresponding to the censored values. How large must $n$ become before that area becomes inconsequentially small? $\endgroup$
    – whuber
    Commented Mar 16, 2019 at 16:18
  • $\begingroup$ You're right. I got confused. Actually, I meant that $\sqrt{n}(\hat{\theta}_{CMLE} - \theta^*)$ has a $N(0,1)$ distribution. I'm a little confused. When you say that $\hat{\theta}$ censors the distribution, you refer to $ \bar {X} _n $ or $\hat{\theta}_{CMLE}$? $\endgroup$
    – Fam
    Commented Mar 16, 2019 at 17:12
  • 1
    $\begingroup$ Look at your definition of $\hat \theta:$ when ${\bar X}_n$ is less than $0,$ you replace it by $0.$ That's what (left) censoring is. $\endgroup$
    – whuber
    Commented Mar 16, 2019 at 18:35

1 Answer 1

1
$\begingroup$

If you run maximum likelihood estimation while imposing the constraint $\theta^* >0$, and if $\theta^* \sim 0$, then you have the situation where the true value of the parameter lies virtually on the boundary of the parameter space.

In such a case, $\hat{\theta}_{CMLE}$ will approach $\theta^{*}$ predominantly from above. It will then take a very-very large number of samples to obtain the asymptotic normality result (and this is why you were told that "you will not have a good approximation"), while for sample sizes that usually suffice for a good approximation, in this case they will not be enough, because then the values of the MLE will not be symmetrically distributed around the true value.

There exist theoretical results as regards the behavior and properties of the MLE when the true value of the parameter lies on the boundary of the parameter space (or in practice when the true value is very close to the boundary), see https://stats.stackexchange.com/a/68866/28746

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.