8
$\begingroup$

Let $X_1,..., X_n $ be a random sample of a variable with PDF:

$$f(x|\theta)=\frac{\theta}{x^2} I_{(\theta, \infty)}(x), \theta >0$$

Find the maximum likelihood estimator for $\theta$ and $ E[X]$

My attempt:

The likelihood function is:

$$L(\theta;x) = \theta^n I_{(\theta, \infty)}(x_{(1)}) \prod \frac{1}{x_i^2} = \theta^n I_{(0, x_{(1)}}(\theta) \prod \frac{1}{x_i^2}$$

Since the indicator function and the product are positive, the likelihood function is increasing. Also, since $\theta$ is on the interval given in the indicator, then $\theta$ is maximum when $\theta = X_{(1)}$. (Is this correct?)

The second doubt is about how to find an estimator of $E[X]$. Calculating it, we have that

$$E[X] = \infty$$

What should I answer, in this case? Or did I do something wrong?

Improve this question
$\endgroup$
12
  • $\begingroup$ What is the support of $x$? $\endgroup$
    – Dan
    Commented Sep 4, 2014 at 19:39
  • 1
    $\begingroup$ The interval $(\theta, \infty)$. See the indicator. $\endgroup$
    – Giiovanna
    Commented Sep 4, 2014 at 19:41
  • 1
    $\begingroup$ It is interesting that your two uses of "$\infty$" differ, @Dan. The first one means the improper integral diverges whereas the second one means the improper integral fails to converge. In some cases the distinction could be important. $\endgroup$
    – whuber ♦
    Commented Sep 4, 2014 at 20:35
  • 3
    $\begingroup$ The expectation of the Cauchy is undefined. It's not $\infty$. $\endgroup$
    – Zen
    Commented Sep 4, 2014 at 20:44
  • 2
    $\begingroup$ @Dan The Cauchy doesn't have $E(X)=\infty$, taking the limit is like evaluating $\lim_{a,b\to\infty} a-b$ -- it is simply undefined (depending on how you arrange to take the two limits you could make it come out to anything at all, so it doesn't have a defined limit). $\endgroup$
    – Glen_b
    Commented Sep 4, 2014 at 22:33

2 Answers 2

Reset to default
8
$\begingroup$

Let $X_1,\dots,X_n$ be a random sample from density $f(x_i)=(\theta/x_i^2)\,I_{(\theta,\infty)}(x_i)$, for $\theta>0$. Since $I_{(\theta,\infty)}(x_i)=I_{(0, x_i)}(\theta)$, writing $x=(x_1,\dots,x_n)$, the likelihood function is $$ L_x(\theta) = \frac{\theta^n}{\prod_{i=1}^n x_i^2} I_{(0,x_{(1)})}(\theta) \, , \qquad (*) $$ in which $x_{(1)}=\min\{x_1,\dots,x_n\}$. The way the density is defined implies that there is no MLE in the usual sense, because the candidate $x_{(1)}\neq\arg\max_\theta L_x(\theta)$. In fact, $L_x(x_{(1)})=0$. If, for each $\theta>0$, we change the version of the density in just one point, and this doesn't change the family of sampling distributions, doing $f(x_i)=(\theta/x_i^2)\,I_{[\theta,\infty)}(x_i)$, then it's true that $\hat{\theta}_{\mathrm{MLE}}=X_{(1)}$.

This is not a serious difficulty, but it's a curious case in which the particular versions of the sampling densities chosen for the problem change the answer. In the second edition of DeGroot's "Probability and Statistics" there is a similar example starting on page 343.

enter image description here

Also, since $\mathrm{E}_\theta[X_i]=\infty$, for every $\theta>0$, asking for an MLE of this quantity doesn't make sense.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ @Zen I am not seeing the problem. Apparently your point hinges on whether the law for $\theta$ assigns zero density to the value $\theta$ itself, but this does not matter since the PDF is only defined modulo functions on sets of measure zero in the first place. Thus this issue seems to be one of notation rather than suggesting any inherent difficulty. Life would be a little tougher if the space of possible $\theta$ were more limited and did not actually include $\theta=x_{(1)}$, for then either form (max or sup) of the MLE would stipulate an invalid value of $\theta$. $\endgroup$
    – whuber ♦
    Commented Sep 4, 2014 at 20:56
  • $\begingroup$ Indeed we use supremum in my course, so we do have a MLE, in this case. $\endgroup$
    – Giiovanna
    Commented Sep 4, 2014 at 21:39
  • 1
    $\begingroup$ One natural way to cope with the situation in the quotation is to recognize that all data are actually intervals, because few measurements are absolutely perfect. Thus in reality the likelihood of $x$ would not be $f(x|\theta)$ but rather $F(x+\epsilon_{+}|\theta)-F(x-\epsilon_{-}|\theta)$ for appropriately small $\epsilon_{+,-}$. One could approach the solution of the original (non-interval) problem by studying the limiting MLE as a function of all these $\epsilon$: there is a limit as they approach zero and it equals $\max(x_1,\ldots,x_n)$. $\endgroup$
    – whuber ♦
    Commented Sep 5, 2014 at 17:17
2
$\begingroup$

One cannot provide an estimator for a distribution moment that is not finite. Here we have a case of a distribution that does not have a finite expected value. The question that usually comes next is "then what does the sample mean from an i.i.d. sample estimates in such a case?" The answer is that the sample mean is a linear function of the random variables of the sample, and it too, has no finite expected value. So it is not an estimator of the expected value of the random variable.

In such cases, we look for other centrality measures, like for example the median. Here we have

$$F_X(x) = \int_\theta^{x}\frac {\theta}{t^2}dt = 1-\frac {\theta}{x}$$

and denoting the median by $m$ we get

$$F_X(m) = \frac 12 \Rightarrow 1-\frac {\theta}{m} = \frac 12 \Rightarrow m=2\theta$$

Therefore an MLE for a centrality measure of this distribution is

$$\hat m_{MLE} = 2\hat \theta_{MLE} = 2X_{(1)}$$

Improve this answer
$\endgroup$
4
  • $\begingroup$ A statistic does not have to have an expectation in order to be an estimator. For instance, the mean of an iid sample from a Cauchy distribution estimates the median. It just happens to be a terrible estimator. In fact, one could use the sample mean to estimate literally any property of the underlying distribution--including a property that is well-defined but infinite. There are various ways to assess how well the sample mean performs that function, such as expected loss. $\endgroup$
    – whuber ♦
    Commented Jun 25, 2015 at 1:58
  • $\begingroup$ @whuber I have trouble understanding the meaning of "estimating something that is infinite", with something that is finite. By construction, the distance between the two will always be infinite, isn't that so? $\endgroup$
    – Alecos Papadopoulos
    Commented Jun 25, 2015 at 2:23
  • 2
    $\begingroup$ It depends on the loss function! That's a key idea to the theory of estimation. For instance, suppose I assume a sample is iid from a location-shifted Student t distribution with unknown degrees of freedom df of $1$ or greater and unknown amount of shifting. I wish to estimate the amount of shifting, but I care more about small amounts than large. My loss for estimating $\delta^{*}$ when the true shift is $\delta$ might be (say) $|\arctan(\delta^{*})-\arctan(\delta)|$. Even when $\delta^{*}$ is the sample mean and df=1, this has a finite expectation. $\endgroup$
    – whuber ♦
    Commented Jun 25, 2015 at 2:30
  • $\begingroup$ @whuber Great example -I think I am beginning to understand. Thanks! $\endgroup$
    – Alecos Papadopoulos
    Commented Jun 25, 2015 at 2:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.