Consider i.i.d random variables $X_1$, $X_2$, . . . , $X_n$ having pdf

$$f_X(x\mid\theta) = \begin{cases} \theta x^{−2} & x\geq\theta \\ 0 & x\lt\theta \end{cases}$$

where $\theta \gt 0$ and $n\geq 3$

(a) Give the likelihood function, expressing it clearly as a function of $\theta$

(b) Give the MSE of the MLE of $\theta$

My Attempt:

(a) $$L(\theta\mid\vec{x}) = \begin{cases} \theta^n\left(\prod_{i=1}^n x_i\right)^{−2} & x_{(1)}\geq\theta \\ 0 & x_{(1)}\lt\theta \end{cases}$$

(b) Clearly the MLE of $\theta$ is $X_{(1)}$. We have

$$\begin{align*} F_{X_{(1)}}(x) &=\mathsf P(\text{min}{\{X_1,...,X_n}\}\leq x)\\\\ &=1-\mathsf P(\text{min}{\{X_1,...,X_n}\}\gt x)\\\\ &=1-\left(1-F_X(x)\right)^n\\\\ &=1-\left(\frac{\theta}{x}\right)^n \end{align*}$$



It follows that

$$\begin{align*} \mathsf E\left(X_{(1)}^2\right) &=\int_{\theta}^{\infty}\frac{n\theta^n}{x^{n-1}}dx\\\\ &=n\theta^n\left(\frac{x^{-n+2}}{-n+2}\Biggr{|}_{\theta}^{\infty}\right)\\\\ &=\frac{n\theta^2}{n-2} \end{align*}$$


$$\begin{align*} \mathsf E\left(X_{(1)}\right) &=\int_{\theta}^{\infty}\frac{n\theta^n}{x^{n}}dx\\\\ &=n\theta^n\left(\frac{x^{-n+1}}{-n+1}\Biggr{|}_{\theta}^{\infty}\right)\\\\ &=\frac{n\theta}{n-1} \end{align*}$$


$$\begin{align*} \mathsf{Var}\left(X_{(1)}\right) &=\frac{n\theta^2}{n-2}-\left(\frac{n\theta}{n-1}\right)^2\\\\ &=\theta^2\left(\frac{n}{n-2}-\frac{n^2}{(n-1)^2}\right) \end{align*}$$

We also have

$$\begin{align*} \text{bias}^2\left(\hat{\theta}\right) &=\left(\mathsf E\left(\hat{\theta}\right)-\theta\right)^2\\\\ &=\left(\frac{n\theta}{n-1}-\theta\right)^2\\\\ &=\left(\theta\left(\frac{n}{n-1}-1\right)\right)^2 \end{align*}$$

Finally the MSE is given by

$$\begin{align*} \mathsf{Var}\left(X_{(1)}\right)+\text{bias}^2\left(\hat{\theta}\right) &=\theta^2\left(\frac{n}{n-2}-\frac{n^2}{(n-1)^2}\right)+\left(\theta\left(\frac{n}{n-1}-1\right)\right)^2\\\\ &=\theta^2\left(\frac{n}{n-2}-\frac{n^2}{(n-1)^2}+\left(\frac{n}{n-1}-1\right)^2\right)\\\\ &=\frac{2\theta^2}{(n-1)(n-2)} \end{align*}$$

Are these valid solutions?

  • 6
    $\begingroup$ You have followed the correct steps. You can check your computations by noting that the population pdf is a Pareto density. And so is the pdf of $X_{(1)}$. $\endgroup$ – StubbornAtom Nov 8 '18 at 20:01

This question is now old enough to give a full succinct solution confirming your calculations. Using standard notation for order statistics, the likelihood function here is:

$$\begin{aligned} L_\mathbf{x}(\theta) &= \prod_{i=1}^n f_X(x_i|\theta) \\[6pt] &= \prod_{i=1}^n \frac{\theta}{x_i^2} \cdot \mathbb{I}(x_i \geqslant) \\[6pt] &\propto \prod_{i=1}^n \theta \cdot \mathbb{I}(x_i \geqslant \theta) \\[12pt] &= \theta^n \cdot \mathbb{I}(0 < \theta \leqslant x_{(1)}). \\[6pt] \end{aligned}$$

This function is strictly increasing over the range $0 < \theta \leqslant x_{(1)}$ so the MLE is:

$$\hat{\theta} = x_{(1)}.$$

Mean-squared-error of MLE: Rather than deriving the distribution of the estimator, it is quicker in this case to derive the distribution of the estimation error. Define the estimation error as $T \equiv \hat{\theta} - \theta$ and note that it has distribution function:

$$\begin{aligned} F_T(t) \equiv \mathbb{P}(\hat{\theta} - \theta \leqslant t) &= 1-\mathbb{P}(\hat{\theta} > \theta + t) \\[6pt] &= 1-\prod_{i=1}^n \mathbb{P}(X_i > \theta + t) \\[6pt] &= 1-(1-F_X(\theta + t))^n \\[6pt] &= \begin{cases} 0 & & \text{for } t < 0, \\[6pt] 1 - \Big( \frac{\theta}{\theta + t} \Big)^n & & \text{for } t \geqslant 0. \\[6pt] \end{cases} \end{aligned}$$

Thus, the density has support over $t \geqslant 0$, where we have:

$$\begin{aligned} f_T(t) \equiv \frac{d F_T}{dt}(t) &= - n \Big( - \frac{\theta}{(\theta + t)^2} \Big) \Big( \frac{\theta}{\theta + t} \Big)^{n-1} \\[6pt] &= \frac{n \theta^n}{(\theta + t)^{n+1}}. \\[6pt] \end{aligned}$$

Assuming that $n>2$, the mean-squared error of the estimator is therefore given by:

$$\begin{aligned} \text{MSE}(\hat{\theta}) = \mathbb{E}(T^2) &= \int \limits_0^\infty t^2 \frac{n \theta^n}{(\theta + t)^{n+1}} \ dt \\[6pt] &= n \theta^n \int \limits_0^\infty \frac{t^2}{(\theta + t)^{n+1}} \ dt \\[6pt] &= n \theta^n \int \limits_\theta^\infty \frac{(r-\theta)^2}{r^{n+1}} \ dr \\[6pt] &= n \theta^n \int \limits_\theta^\infty \Big[ r^{-(n-1)} - 2 \theta r^{-n} + \theta^2 r^{-(n+1)} \Big] \ dr \\[6pt] &= n \theta^n \Bigg[ -\frac{r^{-(n-2)}}{n-2} + \frac{2 \theta r^{-(n-1)}}{n-1} - \frac{\theta^2 r^{-n}}{n} \Bigg]_{r = \theta}^{r \rightarrow \infty} \\[6pt] &= n \theta^n \Bigg[ \frac{\theta^{-(n-2)}}{n-2} - \frac{2 \theta^{-(n-2)}}{n-1} + \frac{\theta^{-(n-2)}}{n} \Bigg] \\[6pt] &= n \theta^2 \Bigg[ \frac{1}{n-2} - \frac{2}{n-1} + \frac{1}{n} \Bigg] \\[6pt] &= \theta^2 \cdot \frac{n(n-1) - 2n(n-2) + (n-1)(n-2)}{(n-1)(n-2)} \\[6pt] &= \theta^2 \cdot \frac{n^2 - n - 2n^2 + 4n + n^2 - 3n + 2}{(n-1)(n-2)} \\[6pt] &= \frac{2\theta^2}{(n-1)(n-2)}. \\[6pt] \end{aligned}$$

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.