Consider i.i.d random variables $X_1$, $X_2$, . . . , $X_n$ having pdf

$$f_X(x\mid\theta) = \begin{cases} \theta x^{−2} & x\geq\theta \\ 0 & x\lt\theta \end{cases}$$

where $\theta \gt 0$ and $n\geq 3$

(a) Give the likelihood function, expressing it clearly as a function of $\theta$

(b) Give the MSE of the MLE of $\theta$

My Attempt:

(a) $$L(\theta\mid\vec{x}) = \begin{cases} \theta^n\left(\prod_{i=1}^n x_i\right)^{−2} & x_{(1)}\geq\theta \\ 0 & x_{(1)}\lt\theta \end{cases}$$

(b) Clearly the MLE of $\theta$ is $X_{(1)}$. We have

$$\begin{align*} F_{X_{(1)}}(x) &=\mathsf P(\text{min}{\{X_1,...,X_n}\}\leq x)\\\\ &=1-\mathsf P(\text{min}{\{X_1,...,X_n}\}\gt x)\\\\ &=1-\left(1-F_X(x)\right)^n\\\\ &=1-\left(\frac{\theta}{x}\right)^n \end{align*}$$



It follows that

$$\begin{align*} \mathsf E\left(X_{(1)}^2\right) &=\int_{\theta}^{\infty}\frac{n\theta^n}{x^{n-1}}dx\\\\ &=n\theta^n\left(\frac{x^{-n+2}}{-n+2}\Biggr{|}_{\theta}^{\infty}\right)\\\\ &=\frac{n\theta^2}{n-2} \end{align*}$$


$$\begin{align*} \mathsf E\left(X_{(1)}\right) &=\int_{\theta}^{\infty}\frac{n\theta^n}{x^{n}}dx\\\\ &=n\theta^n\left(\frac{x^{-n+1}}{-n+1}\Biggr{|}_{\theta}^{\infty}\right)\\\\ &=\frac{n\theta}{n-1} \end{align*}$$


$$\begin{align*} \mathsf{Var}\left(X_{(1)}\right) &=\frac{n\theta^2}{n-2}-\left(\frac{n\theta}{n-1}\right)^2\\\\ &=\theta^2\left(\frac{n}{n-2}-\frac{n^2}{(n-1)^2}\right) \end{align*}$$

We also have

$$\begin{align*} \text{bias}^2\left(\hat{\theta}\right) &=\left(\mathsf E\left(\hat{\theta}\right)-\theta\right)^2\\\\ &=\left(\frac{n\theta}{n-1}-\theta\right)^2\\\\ &=\left(\theta\left(\frac{n}{n-1}-1\right)\right)^2 \end{align*}$$

Finally the MSE is given by

$$\begin{align*} \mathsf{Var}\left(X_{(1)}\right)+\text{bias}^2\left(\hat{\theta}\right) &=\theta^2\left(\frac{n}{n-2}-\frac{n^2}{(n-1)^2}\right)+\left(\theta\left(\frac{n}{n-1}-1\right)\right)^2\\\\ &=\theta^2\left(\frac{n}{n-2}-\frac{n^2}{(n-1)^2}+\left(\frac{n}{n-1}-1\right)^2\right)\\\\ &=\frac{2\theta^2}{(n-1)(n-2)} \end{align*}$$

Are these valid solutions?

  • 6
    $\begingroup$ You have followed the correct steps. You can check your computations by noting that the population pdf is a Pareto density. And so is the pdf of $X_{(1)}$. $\endgroup$ Commented Nov 8, 2018 at 20:01

1 Answer 1


This question is now old enough to give a full succinct solution confirming your calculations. Using standard notation for order statistics, the likelihood function here is:

$$\begin{aligned} L_\mathbf{x}(\theta) &= \prod_{i=1}^n f_X(x_i|\theta) \\[6pt] &= \prod_{i=1}^n \frac{\theta}{x_i^2} \cdot \mathbb{I}(x_i \geqslant) \\[6pt] &\propto \prod_{i=1}^n \theta \cdot \mathbb{I}(x_i \geqslant \theta) \\[12pt] &= \theta^n \cdot \mathbb{I}(0 < \theta \leqslant x_{(1)}). \\[6pt] \end{aligned}$$

This function is strictly increasing over the range $0 < \theta \leqslant x_{(1)}$ so the MLE is:

$$\hat{\theta} = x_{(1)}.$$

Mean-squared-error of MLE: Rather than deriving the distribution of the estimator, it is quicker in this case to derive the distribution of the estimation error. Define the estimation error as $T \equiv \hat{\theta} - \theta$ and note that it has distribution function:

$$\begin{aligned} F_T(t) \equiv \mathbb{P}(\hat{\theta} - \theta \leqslant t) &= 1-\mathbb{P}(\hat{\theta} > \theta + t) \\[6pt] &= 1-\prod_{i=1}^n \mathbb{P}(X_i > \theta + t) \\[6pt] &= 1-(1-F_X(\theta + t))^n \\[6pt] &= \begin{cases} 0 & & \text{for } t < 0, \\[6pt] 1 - \Big( \frac{\theta}{\theta + t} \Big)^n & & \text{for } t \geqslant 0. \\[6pt] \end{cases} \end{aligned}$$

Thus, the density has support over $t \geqslant 0$, where we have:

$$\begin{aligned} f_T(t) \equiv \frac{d F_T}{dt}(t) &= - n \Big( - \frac{\theta}{(\theta + t)^2} \Big) \Big( \frac{\theta}{\theta + t} \Big)^{n-1} \\[6pt] &= \frac{n \theta^n}{(\theta + t)^{n+1}}. \\[6pt] \end{aligned}$$

Assuming that $n>2$, the mean-squared error of the estimator is therefore given by:

$$\begin{aligned} \text{MSE}(\hat{\theta}) = \mathbb{E}(T^2) &= \int \limits_0^\infty t^2 \frac{n \theta^n}{(\theta + t)^{n+1}} \ dt \\[6pt] &= n \theta^n \int \limits_0^\infty \frac{t^2}{(\theta + t)^{n+1}} \ dt \\[6pt] &= n \theta^n \int \limits_\theta^\infty \frac{(r-\theta)^2}{r^{n+1}} \ dr \\[6pt] &= n \theta^n \int \limits_\theta^\infty \Big[ r^{-(n-1)} - 2 \theta r^{-n} + \theta^2 r^{-(n+1)} \Big] \ dr \\[6pt] &= n \theta^n \Bigg[ -\frac{r^{-(n-2)}}{n-2} + \frac{2 \theta r^{-(n-1)}}{n-1} - \frac{\theta^2 r^{-n}}{n} \Bigg]_{r = \theta}^{r \rightarrow \infty} \\[6pt] &= n \theta^n \Bigg[ \frac{\theta^{-(n-2)}}{n-2} - \frac{2 \theta^{-(n-2)}}{n-1} + \frac{\theta^{-(n-2)}}{n} \Bigg] \\[6pt] &= n \theta^2 \Bigg[ \frac{1}{n-2} - \frac{2}{n-1} + \frac{1}{n} \Bigg] \\[6pt] &= \theta^2 \cdot \frac{n(n-1) - 2n(n-2) + (n-1)(n-2)}{(n-1)(n-2)} \\[6pt] &= \theta^2 \cdot \frac{n^2 - n - 2n^2 + 4n + n^2 - 3n + 2}{(n-1)(n-2)} \\[6pt] &= \frac{2\theta^2}{(n-1)(n-2)}. \\[6pt] \end{aligned}$$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.