Finding MLE and MSE of $\theta$ where $f_X(x\mid\theta)=\theta x^{−2} I_{x\geq\theta}(x)$

Question

Consider i.i.d random variables $X_1$, $X_2$, . . . , $X_n$ having pdf

$$f_X(x\mid\theta) = \begin{cases} \theta x^{−2} & x\geq\theta \\ 0 & x\lt\theta \end{cases}$$

where $\theta \gt 0$ and $n\geq 3$

(a) Give the likelihood function, expressing it clearly as a function of $\theta$

(b) Give the MSE of the MLE of $\theta$

My Attempt:

(a) $$L(\theta\mid\vec{x}) = \begin{cases} \theta^n\left(\prod_{i=1}^n x_i\right)^{−2} & x_{(1)}\geq\theta \\ 0 & x_{(1)}\lt\theta \end{cases}$$

(b) Clearly the MLE of $\theta$ is $X_{(1)}$. We have

$$\begin{align*} F_{X_{(1)}}(x) &=\mathsf P(\text{min}{\{X_1,...,X_n}\}\leq x)\\\\ &=1-\mathsf P(\text{min}{\{X_1,...,X_n}\}\gt x)\\\\ &=1-\left(1-F_X(x)\right)^n\\\\ &=1-\left(\frac{\theta}{x}\right)^n \end{align*}$$

so

$$f_{X_{(1)}}(x)=\left(1-\left(\frac{\theta}{x}\right)^n\right)'=\frac{n\theta^n}{x^{n+1}}I_{[\theta,\infty)}(x)$$

It follows that

$$\begin{align*} \mathsf E\left(X_{(1)}^2\right) &=\int_{\theta}^{\infty}\frac{n\theta^n}{x^{n-1}}dx\\\\ &=n\theta^n\left(\frac{x^{-n+2}}{-n+2}\Biggr{|}_{\theta}^{\infty}\right)\\\\ &=\frac{n\theta^2}{n-2} \end{align*}$$

and

$$\begin{align*} \mathsf E\left(X_{(1)}\right) &=\int_{\theta}^{\infty}\frac{n\theta^n}{x^{n}}dx\\\\ &=n\theta^n\left(\frac{x^{-n+1}}{-n+1}\Biggr{|}_{\theta}^{\infty}\right)\\\\ &=\frac{n\theta}{n-1} \end{align*}$$

so

$$\begin{align*} \mathsf{Var}\left(X_{(1)}\right) &=\frac{n\theta^2}{n-2}-\left(\frac{n\theta}{n-1}\right)^2\\\\ &=\theta^2\left(\frac{n}{n-2}-\frac{n^2}{(n-1)^2}\right) \end{align*}$$

We also have

$$\begin{align*} \text{bias}^2\left(\hat{\theta}\right) &=\left(\mathsf E\left(\hat{\theta}\right)-\theta\right)^2\\\\ &=\left(\frac{n\theta}{n-1}-\theta\right)^2\\\\ &=\left(\theta\left(\frac{n}{n-1}-1\right)\right)^2 \end{align*}$$

Finally the MSE is given by

$$\begin{align*} \mathsf{Var}\left(X_{(1)}\right)+\text{bias}^2\left(\hat{\theta}\right) &=\theta^2\left(\frac{n}{n-2}-\frac{n^2}{(n-1)^2}\right)+\left(\theta\left(\frac{n}{n-1}-1\right)\right)^2\\\\ &=\theta^2\left(\frac{n}{n-2}-\frac{n^2}{(n-1)^2}+\left(\frac{n}{n-1}-1\right)^2\right)\\\\ &=\frac{2\theta^2}{(n-1)(n-2)} \end{align*}$$

Are these valid solutions?

Answer 1

This question is now old enough to give a full succinct solution confirming your calculations. Using standard notation for order statistics, the likelihood function here is:

$$\begin{aligned} L_\mathbf{x}(\theta) &= \prod_{i=1}^n f_X(x_i|\theta) \\[6pt] &= \prod_{i=1}^n \frac{\theta}{x_i^2} \cdot \mathbb{I}(x_i \geqslant) \\[6pt] &\propto \prod_{i=1}^n \theta \cdot \mathbb{I}(x_i \geqslant \theta) \\[12pt] &= \theta^n \cdot \mathbb{I}(0 < \theta \leqslant x_{(1)}). \\[6pt] \end{aligned}$$

This function is strictly increasing over the range $0 < \theta \leqslant x_{(1)}$ so the MLE is:

$$\hat{\theta} = x_{(1)}.$$

---

Mean-squared-error of MLE: Rather than deriving the distribution of the estimator, it is quicker in this case to derive the distribution of the estimation error. Define the estimation error as $T \equiv \hat{\theta} - \theta$ and note that it has distribution function:

$$\begin{aligned} F_T(t) \equiv \mathbb{P}(\hat{\theta} - \theta \leqslant t) &= 1-\mathbb{P}(\hat{\theta} > \theta + t) \\[6pt] &= 1-\prod_{i=1}^n \mathbb{P}(X_i > \theta + t) \\[6pt] &= 1-(1-F_X(\theta + t))^n \\[6pt] &= \begin{cases} 0 & & \text{for } t < 0, \\[6pt] 1 - \Big( \frac{\theta}{\theta + t} \Big)^n & & \text{for } t \geqslant 0. \\[6pt] \end{cases} \end{aligned}$$

Thus, the density has support over $t \geqslant 0$, where we have:

$$\begin{aligned} f_T(t) \equiv \frac{d F_T}{dt}(t) &= - n \Big( - \frac{\theta}{(\theta + t)^2} \Big) \Big( \frac{\theta}{\theta + t} \Big)^{n-1} \\[6pt] &= \frac{n \theta^n}{(\theta + t)^{n+1}}. \\[6pt] \end{aligned}$$

Assuming that $n>2$, the mean-squared error of the estimator is therefore given by:

$$\begin{aligned} \text{MSE}(\hat{\theta}) = \mathbb{E}(T^2) &= \int \limits_0^\infty t^2 \frac{n \theta^n}{(\theta + t)^{n+1}} \ dt \\[6pt] &= n \theta^n \int \limits_0^\infty \frac{t^2}{(\theta + t)^{n+1}} \ dt \\[6pt] &= n \theta^n \int \limits_\theta^\infty \frac{(r-\theta)^2}{r^{n+1}} \ dr \\[6pt] &= n \theta^n \int \limits_\theta^\infty \Big[ r^{-(n-1)} - 2 \theta r^{-n} + \theta^2 r^{-(n+1)} \Big] \ dr \\[6pt] &= n \theta^n \Bigg[ -\frac{r^{-(n-2)}}{n-2} + \frac{2 \theta r^{-(n-1)}}{n-1} - \frac{\theta^2 r^{-n}}{n} \Bigg]_{r = \theta}^{r \rightarrow \infty} \\[6pt] &= n \theta^n \Bigg[ \frac{\theta^{-(n-2)}}{n-2} - \frac{2 \theta^{-(n-2)}}{n-1} + \frac{\theta^{-(n-2)}}{n} \Bigg] \\[6pt] &= n \theta^2 \Bigg[ \frac{1}{n-2} - \frac{2}{n-1} + \frac{1}{n} \Bigg] \\[6pt] &= \theta^2 \cdot \frac{n(n-1) - 2n(n-2) + (n-1)(n-2)}{(n-1)(n-2)} \\[6pt] &= \theta^2 \cdot \frac{n^2 - n - 2n^2 + 4n + n^2 - 3n + 2}{(n-1)(n-2)} \\[6pt] &= \frac{2\theta^2}{(n-1)(n-2)}. \\[6pt] \end{aligned}$$