The MLE being $X_{1:n}$ where $1:n$ indicates the minimum value. The pdf of the distribution is given as: $$ f(x;\theta)=\begin{cases}2\theta^2x^{-3}\hspace{10pt}\theta\leq{x}\\0\hspace{33pt}x<\theta;0<\theta\end{cases} $$ Calculating the log-likelihood gives $$ \ell(\theta)=n\ln(2)+2n\ln(\theta)-3\sum^n_{i=1}\ln(x_i), $$ Leading to concluding that the Log likelihood will minimize if $\theta$ rises so we have a max L if $\hat{\theta}=x_n$. What I don't understand is how I can show that this MLE is biased.
2 Answers
First, note that the CDF of the minimum of $n$ iid observations from a continuous variable is
\begin{align*} F_{X_{1:n}}(x) &= Pr(\min(X_1, \dots,X_n) \leq x)\\ &= (1 - Pr(\min(X_1, \dots,X_n) > x))\\ &= (1 - \prod_{i=1}^n Pr(X_i > x))\\ &= 1 - (1 - F_{X}(x))^n. \end{align*}
By differentiating this with respect to $x$, we find that the pdf of the minimum is
\begin{align*} f_{X_{1:n}}(x) &= n(1 - F_{X}(x))^{n-1}f_X(x). \end{align*}
By integrating the pdf of each $X_i$ we can find that \begin{align*} F_X(x) &= \int_{\theta}^x f_X(t)dt\\ &= \int_{\theta}^x 2\theta^2t^{-3}dt\\ &= 1 - \frac{\theta^2}{x^2}, \; x \geq \theta, \end{align*} so therefore we can find that the pdf of the minimum is
\begin{align*} f_{X_{1:n}}(x) &= n\left(\frac{\theta^2}{x^2}\right)^{n-1}2\theta^2x^{-3}\\ &= 2n\frac{\theta^{2n}}{x^{2n + 1}}. \end{align*}
Now we can compute that the mean of the minimum is
\begin{align*} \int_{\theta}^\infty x 2n\frac{\theta^{2n}}{x^{2n + 1}}dx &= \int_{\theta}^\infty 2n\frac{\theta^{2n}}{x^{2n}}dx\\ &= \frac{2n}{2n - 1}\frac{\theta^{2n}}{\theta^{2n - 1}}\\ &= \frac{1}{1 - \frac{1}{2n}}\theta. \end{align*} So, the maximum likelihood estimator $\min(X_1, \dots, X_n)$ is biased by the factor $\frac{1}{1 - \frac{1}{2n}}.$
-
1$\begingroup$ I've been trying to find the unbiasing factor and just figured out its $\frac{2n}{2n-1}.$ Checking my math once more before posting. But it's the same as your answer. (+1) $\endgroup$– BruceETOct 12, 2019 at 23:59
-
1$\begingroup$ Please note the help center in relation to homework style questions. $\endgroup$– Glen_bOct 13, 2019 at 0:27
You have a Pareto distribution.
Outline of solution: (a) Identify parameters and write the CDF. (b) Use the CDF method to find the CDF of $X_{1:n} = \min_i(X_1, \dots, X_n).$ (c) Find PDF of the minimum from its CDF. (c) Find expectation.
Graph of density of $X_i,$ for $\theta=2.$
As you say, the MLE is $V = X_{1:n} = \min_i(X_1, \dots, X_n).$ But in practice, the true $\theta$ must be (at least a tiny bit) smaller than the minimum from data, hence the bias. Using Wikipedia's notation, your particular Pareto distribution has $x_m = \theta$ for the scale parameter and $\alpha = 2$ for the (known) shape parameter.
The distribution of the minimum of $n$ observations is again Pareto with the same scale parameter $\theta$ as the data and shape parameter $n\alpha.$ Then because the Pareto mean is $\mu = \alpha\theta/(\alpha - 1),$ the unbiasing constant for your distribution is $c = 2n/(2n-1).$ This agrees with @SimonBogeBrant's Answer (+1).
Because your Pareto random variable can be simulated as $X = \theta/\sqrt{U},$ where $U \sim \mathsf{Unif}(0,1),$ it is easy to simulate the minimum of $n$ of your Pareto random variables and illustrate this result. [The Wikipedia link has a section on simulating Pareto distributions. I am using the base of R which doesn't have tailor-make functions for Pareto distributions.]
set.seed(1210)
n = 10; th=2
v = replicate(10^6, min(th/sqrt(runif(n))))
mean(v)
[1] 2.105398 # aprx E(V) > 2
c = (2*n-1)/(2*n)
mean(c*v)
[1] 2.000128 # aprx E(cV) = 2
hist(v, prob=T, br=20, col="skyblue2")
Notes: (1) Runs of the program with several different sample sizes $n$ gave similarly good results.
(2) Even though another answer has a completely worked version of this problem, I hope you will follow through my outline and some relevant parts of the Wikipedia article to get familiar with Pareto distributions and to make sure you understand and remember how to do this general type of problem.
-
1$\begingroup$ A simpler approach is to stop after finding the CDF of $X_{1:n}$, note that $X_{1:n} \geq \theta$ and that (from the CDF) the probability that it is $> \theta$ is $> 0$; these two facts combined are sufficient to show that $\mathbb{E}X_{1:n} \geq \theta$ and that therefore it is biased. $\endgroup$– jbowmanOct 12, 2019 at 23:05
-
$\begingroup$ @jbowman. Thanks for comment. Clearly biased as you say. But I've been trying tind unbiasing constant (amongst interruptions). Think I've got it. If so, will illustrate in a simulation. Just noticed. Looks as if another answer already has it. $\endgroup$– BruceETOct 12, 2019 at 23:52