Finding a maximum likelihood estimate of a parameter when regularity conditions don't hold

Question

I am currently trying to find the maximum likelihood estimate (or determine that it doesn't exist) of the parameter $a$ for a random sample of the variables $X_1, X_2 ...X_n$ with pdf $$f(x;a)= \begin{cases} ax^{-2}&\text{if}\, 0 < a \leq x < \infty \\ 0&\text{otherwise} \end{cases}$$

I've determined that the likelihood $L(a;x)$ is $$L(a;x)= \begin{cases} a^n\prod_{i=1}^{n} x_i^{-2}&\text{if}\, \min(x_i) \geq a \\ 0&\text{otherwise} \end{cases}$$

I know that this case doesn't satisfy the regularity conditions for using the derivative to maximize the likelihood, so I'm not sure where to go with this problem.

Answer 1

You are correct that the problem doesn't satisfy the regularity conditions, but you can still find the MLE by looking at the derivative. Let's work with the log likelihood:

$$l(a;x) = n\log(a) -2\Sigma\log(x_i)$$

Clearly this is monotonically increasing in $a$ (this is all the looking at the derivative we're going to do.) So... our MLE $\hat{a}$ will be as large as possible. How large is that? We know $a \leq \min(x_i)$, so the largest $a$ can be is $\min(x_i)$. It follows that $\hat{a} = \min(x_i)$.

Answer 2

The answer by jbowman covers what you need to know to obtain the MLE, and I have nothing to add to that derivation. However, it is worth noting that in cases like this, where you have a sampling density that is monotonic in the parameter, it is usual for the maximum-likelihood estimator (MLE) to be bounded by the true parameter value and so you obtain a biased estimator. In this particular case you have $\hat{a} \geqslant a$ so the MLE will be biased upward. In cases like this it is useful to derive the distribution of the MLE and have a look at its expected value, to quantify the bias. You might also consider adjusting the estimator to correct the bias.

Letting $\hat{A} = \hat{A}(X_1, ..., X_n) = \min X_i$ be the estimator, and assuming for our derivation that $n>2$, we have the distribution function:

$$\begin{equation} \begin{aligned} F_{\hat{A}}(r) \equiv \mathbb{P}(\hat{A} \leqslant r) = \mathbb{P}(\min X_i \leqslant r) &= 1- \mathbb{P}(\min X_i > r) \\[6pt] &= 1-\mathbb{P}(X_1 > r) \cdot ... \cdot \mathbb{P}(X_n > r) \\[6pt] &= \begin{cases} 1-a^n r^{-n} & \text{ for } r \geqslant a, \\[6pt] 0 & \text{ for } r < a, \end{cases} \end{aligned} \end{equation}$$

and corresponding density function:

$$\begin{equation} \begin{aligned} f_{\hat{A}}(r) = \begin{cases} n a^n r^{-n-1} & \text{ for } r \geqslant a, \\[6pt] 0 & \text{ for } r < a. \end{cases} \end{aligned} \end{equation}$$

This gives the expected value:

$$\begin{equation} \begin{aligned} \mathbb{E}(\hat{A}) = \int \limits_a^\infty r f_{\hat{A}}(r) dr &= \int \limits_a^\infty n a^n r^{-n} dr \\[8pt] &= \Bigg[ \frac{n}{n-1} \cdot a^n r^{-(n-1)} \Bigg]_{r=a}^{r \rightarrow \infty} \\[8pt] &= \frac{n}{n-1} \cdot a. \end{aligned} \end{equation}$$

---

Correcting for bias: We can see from the above result that the MLE is biased upward, but it can easily be "corrected" by using the corresponding estimator:

$$A^* \equiv \frac{n-1}{n} \hat{A} = \frac{n-1}{n} \min X_i.$$

With some additional algebra it can be shown that the bias-adjusted MLE has moments:

$$\mathbb{E}(A^*) = a \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \mathbb{V}(A^*) = \frac{a^2}{n(n-2)}.$$

This latter estimator is unbiased and consistent. It is probably a preferable estimator to the MLE.