6
$\begingroup$

Suppose we have i.i.d. $n$ observations $(X_1,X_2,...X_n)$ from a population with density $$f_\theta(x)=\begin{cases}\theta x^{\theta-1}&\text{ if }0\leq x\leq 1\\0&\text{otherwise.}\end{cases}$$ Note that $\theta>0$. If $T_n$ denotes the maximum likelihood estimator of $\theta$ given that $n$ is the sample size, then show that $$\text{var}(T_n)\stackrel{n\to\infty}{\longrightarrow} 0$$

This is the problem I am trying to solve: I computed $$T_n=\dfrac{-n}{\sum_{i=1}^{n}\ln(X_i)}$$ But I am stuck in finding $\text{var}(T_n)$ and showing that $\text{var}(T_n)\to0$ as $n\to\infty$.

Improve this question
$\endgroup$
7
  • 2
    $\begingroup$ can you relate $T_n$ to an empirical average? $\endgroup$
    – Xi'an
    Jan 17, 2015 at 13:21
  • $\begingroup$ I don't quite get your question. I have observed that $T_n$ is the inverse of the A.M. of $\ln(X_i)$ which is trivial to see, along with a negative sign. But we are required to compute the variance of this M.L.E. That's reverting back to population parameters, as I am not interested in the "sample" variance. $\endgroup$
    – Landon Carter
    Jan 17, 2015 at 13:24
  • $\begingroup$ I was suggesting this approach to apply the delta method and deduce from the CLT that the variance of $T_n$ goes to zero. $\endgroup$
    – Xi'an
    Jan 17, 2015 at 13:29
  • $\begingroup$ That seems interesting. So now I need to find a suitable function $g$ to apply the delta method such that $(g(X)-g(\mu))\sqrt{n}\to N(0,\sigma^2(g'(x))^2)$. $\endgroup$
    – Landon Carter
    Jan 17, 2015 at 13:34
  • 3
    $\begingroup$ The transform is straightforward since you start from $\overline{\log(X_i)}$, i.e., $g(y)=1/y$. $\endgroup$
    – Xi'an
    Jan 17, 2015 at 13:39

3 Answers 3

Reset to default
5
$\begingroup$

Combining @Xi'an comments and @wolfies' answer, we have that

$$1/T_n = \frac 1n \sum_{i=1}^n (-\ln X_i)$$

But $-\ln X_i \sim {\rm Exp}(1/\theta)$ (where $1/\theta$ is the scale parameter), which essentially is a Gamma distribution with shape parameter $1$, and so by the summation properties of the Gamma distribution,

$$\sum_{i=1}^n (-\ln X_i) \sim {\rm Gamma}(n, 1/\theta)$$.

Using the scaling properties of the Gamma distribution we obtain

$$1/T_n \sim {\rm Gamma}(n, 1/(n\theta)) $$

Then the inverse of $1/T_n$ (i.e. $T_n$) follows an Inverse Gamma distribution, with same shape paramater and reciprocal scale parameter

$$T_n \sim {\rm InvGamma}(n, n\theta)$$

that has variance

$$ {\rm Var} (T_n) = \frac {n^2\theta^2}{(n-1)^2(n-2)}$$

as the Mathematica software gave.

The leading term in the numerator is $n^2$ while the leading term in the denominator is $n^3$ so the limit of the variance of $T_n$ with respect to $n$ goes to zero.

Improve this answer
$\endgroup$
5
  • $\begingroup$ I think this perhaps goes a fair bit beyond "hints and guidance" suitable for a fresh self-study question. $\endgroup$
    – Glen_b
    Jan 18, 2015 at 5:55
  • $\begingroup$ @Glen_b You are right. I got carried away when I saw another answer providing the result through computation, and though it would be good to have also its analytical dimension, forgetting the self-study tag in the process. $\endgroup$
    – Alecos Papadopoulos
    Jan 18, 2015 at 16:01
  • $\begingroup$ Never mind. In fact, I know nothing about an Inverse Gamma distribution. So had you not written the complete answer, I would have been in the dark. $\endgroup$
    – Landon Carter
    Jan 18, 2015 at 16:09
  • $\begingroup$ The problem with using notation such as InverseGamma(a,b) is that there is no standard definition for what an InverseGamma(a,b) distribution defines. Some authors use IG(a,b), others use IG(a,1/b), and others still other notation ... What is the variance of an undefined object?? Using a lookup table can unfortunately yield the incorrect answer, if the text or lookup table happens to use different defns. Wiki sometimes uses one parameterisation for the pdf, and an inconsistent defn for moments etc (not here) - and they clash (not here). It is best to avoid the ambiguity by defining the pdf. $\endgroup$
    – wolfies
    Jan 18, 2015 at 17:58
  • $\begingroup$ @wolfies This is the same problem we face with the Gamma distribution itself (and with the Exponential too). I have found that mentioning "shape-scale" or "shape-rate" parametrization, appears to resolve the communication problem. $\endgroup$
    – Alecos Papadopoulos
    Jan 18, 2015 at 20:15
4
$\begingroup$

Did you try to compute the Fisher information? Get the second derivative of the Log Likelihood, multiply by -1, and take the reciprocal. Evaluate at the MLE . Asymptotically, this approximates the variance of the MLE.

Improve this answer
$\endgroup$
0
2
$\begingroup$

The Problem

Let $(X_1, \dots, X_n)$ denote a random sample of size $n$ drawn on $$X \sim \text{PowerFunction}(\theta,1) \quad \text{ with pdf} \quad f(x) = \theta x^{\theta-1} \text{ where } \quad 0<x<1.$$ Let $Z = -\sum_{i=1}^{n}\ln(X_i). \quad$ Find: $\quad Var\big[\dfrac{n}{Z}\big]$

Solution

Let $Y = -ln(X)$. Then, by any of the usual methods of transformation, $Y \sim \text{Exponential}(\theta)$ with pdf $\theta e^{-\theta y}$. Next, the sum of $n$ iid Exponentials with rate $\theta$ is well-known to be $\text{Gamma}(n, 1/\theta)$ so $Z \sim \text{Gamma}(n, 1/\theta)$ with pdf, say, $g(z)$:

enter image description here

We seek $Var(\frac{n}{Z})$:

enter image description here

where I am using the Var function from the mathStatica package for Mathematica to automate the calculation.

The limit of the latter tends to 0, as $n \rightarrow \infty$, as required.

Improve this answer
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.