7
$\begingroup$

Let $X_1,..., X_n $ be a random sample of a variable with PDF:

$$f(x|\theta)=\frac{\theta}{x^2} I_{(\theta, \infty)}(x), \theta >0$$

Find the maximum likelihood estimator for $\theta$ and $ E[X]$

My attempt:

The likelihood function is:

$$L(\theta;x) = \theta^n I_{(\theta, \infty)}(x_{(1)}) \prod \frac{1}{x_i^2} = \theta^n I_{(0, x_{(1)}}(\theta) \prod \frac{1}{x_i^2}$$

Since the indicator function and the product are positive, the likelihood function is increasing. Also, since $\theta$ is on the interval given in the indicator, then $\theta$ is maximum when $\theta = X_{(1)}$. (Is this correct?)

The second doubt is about how to find an estimator of $E[X]$. Calculating it, we have that

$$E[X] = \infty$$

What should I answer, in this case? Or did I do something wrong?

$\endgroup$
  • $\begingroup$ What is the support of $x$? $\endgroup$ – Dan Sep 4 '14 at 19:39
  • 1
    $\begingroup$ The interval $(\theta, \infty)$. See the indicator. $\endgroup$ – Giiovanna Sep 4 '14 at 19:41
  • 1
    $\begingroup$ It is interesting that your two uses of "$\infty$" differ, @Dan. The first one means the improper integral diverges whereas the second one means the improper integral fails to converge. In some cases the distinction could be important. $\endgroup$ – whuber Sep 4 '14 at 20:35
  • 3
    $\begingroup$ The expectation of the Cauchy is undefined. It's not $\infty$. $\endgroup$ – Zen Sep 4 '14 at 20:44
  • 2
    $\begingroup$ @Dan The Cauchy doesn't have $E(X)=\infty$, taking the limit is like evaluating $\lim_{a,b\to\infty} a-b$ -- it is simply undefined (depending on how you arrange to take the two limits you could make it come out to anything at all, so it doesn't have a defined limit). $\endgroup$ – Glen_b Sep 4 '14 at 22:33
6
$\begingroup$

Let $X_1,\dots,X_n$ be a random sample from density $f(x_i)=(\theta/x_i^2)\,I_{(\theta,\infty)}(x_i)$, for $\theta>0$. Since $I_{(\theta,\infty)}(x_i)=I_{(0, x_i)}(\theta)$, writing $x=(x_1,\dots,x_n)$, the likelihood function is $$ L_x(\theta) = \frac{\theta^n}{\prod_{i=1}^n x_i^2} I_{(0,x_{(1)})}(\theta) \, , \qquad (*) $$ in which $x_{(1)}=\min\{x_1,\dots,x_n\}$. The way the density is defined implies that there is no MLE in the usual sense, because the candidate $x_{(1)}\neq\arg\max_\theta L_x(\theta)$. In fact, $L_x(x_{(1)})=0$. If, for each $\theta>0$, we change the version of the density in just one point, and this doesn't change the family of sampling distributions, doing $f(x_i)=(\theta/x_i^2)\,I_{[\theta,\infty)}(x_i)$, then it's true that $\hat{\theta}_{\mathrm{MLE}}=X_{(1)}$.

This is not a serious difficulty, but it's a curious case in which the particular versions of the sampling densities chosen for the problem change the answer. In the second edition of DeGroot's "Probability and Statistics" there is a similar example starting on page 343.

enter image description here

Also, since $\mathrm{E}_\theta[X_i]=\infty$, for every $\theta>0$, asking for an MLE of this quantity doesn't make sense.

$\endgroup$
  • $\begingroup$ @Zen I am not seeing the problem. Apparently your point hinges on whether the law for $\theta$ assigns zero density to the value $\theta$ itself, but this does not matter since the PDF is only defined modulo functions on sets of measure zero in the first place. Thus this issue seems to be one of notation rather than suggesting any inherent difficulty. Life would be a little tougher if the space of possible $\theta$ were more limited and did not actually include $\theta=x_{(1)}$, for then either form (max or sup) of the MLE would stipulate an invalid value of $\theta$. $\endgroup$ – whuber Sep 4 '14 at 20:56
  • $\begingroup$ Indeed we use supremum in my course, so we do have a MLE, in this case. $\endgroup$ – Giiovanna Sep 4 '14 at 21:39
  • 1
    $\begingroup$ One natural way to cope with the situation in the quotation is to recognize that all data are actually intervals, because few measurements are absolutely perfect. Thus in reality the likelihood of $x$ would not be $f(x|\theta)$ but rather $F(x+\epsilon_{+}|\theta)-F(x-\epsilon_{-}|\theta)$ for appropriately small $\epsilon_{+,-}$. One could approach the solution of the original (non-interval) problem by studying the limiting MLE as a function of all these $\epsilon$: there is a limit as they approach zero and it equals $\max(x_1,\ldots,x_n)$. $\endgroup$ – whuber Sep 5 '14 at 17:17
2
$\begingroup$

One cannot provide an estimator for a distribution moment that is not finite. Here we have a case of a distribution that does not have a finite expected value. The question that usually comes next is "then what does the sample mean from an i.i.d. sample estimates in such a case?" The answer is that the sample mean is a linear function of the random variables of the sample, and it too, has no finite expected value. So it is not an estimator of the expected value of the random variable.

In such cases, we look for other centrality measures, like for example the median. Here we have

$$F_X(x) = \int_\theta^{x}\frac {\theta}{t^2}dt = 1-\frac {\theta}{x}$$

and denoting the median by $m$ we get

$$F_X(m) = \frac 12 \Rightarrow 1-\frac {\theta}{m} = \frac 12 \Rightarrow m=2\theta$$

Therefore an MLE for a centrality measure of this distribution is

$$\hat m_{MLE} = 2\hat \theta_{MLE} = 2X_{(1)}$$

$\endgroup$
  • $\begingroup$ A statistic does not have to have an expectation in order to be an estimator. For instance, the mean of an iid sample from a Cauchy distribution estimates the median. It just happens to be a terrible estimator. In fact, one could use the sample mean to estimate literally any property of the underlying distribution--including a property that is well-defined but infinite. There are various ways to assess how well the sample mean performs that function, such as expected loss. $\endgroup$ – whuber Jun 25 '15 at 1:58
  • $\begingroup$ @whuber I have trouble understanding the meaning of "estimating something that is infinite", with something that is finite. By construction, the distance between the two will always be infinite, isn't that so? $\endgroup$ – Alecos Papadopoulos Jun 25 '15 at 2:23
  • 2
    $\begingroup$ It depends on the loss function! That's a key idea to the theory of estimation. For instance, suppose I assume a sample is iid from a location-shifted Student t distribution with unknown degrees of freedom df of $1$ or greater and unknown amount of shifting. I wish to estimate the amount of shifting, but I care more about small amounts than large. My loss for estimating $\delta^{*}$ when the true shift is $\delta$ might be (say) $|\arctan(\delta^{*})-\arctan(\delta)|$. Even when $\delta^{*}$ is the sample mean and df=1, this has a finite expectation. $\endgroup$ – whuber Jun 25 '15 at 2:30
  • $\begingroup$ @whuber Great example -I think I am beginning to understand. Thanks! $\endgroup$ – Alecos Papadopoulos Jun 25 '15 at 2:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.