I am currently trying to find the maximum likelihood estimate (or determine that it doesn't exist) of the parameter $a$ for a random sample of the variables $X_1, X_2 ...X_n$ with pdf $$f(x;a)= \begin{cases} ax^{-2}&\text{if}\, 0 < a \leq x < \infty \\ 0&\text{otherwise} \end{cases}$$

I've determined that the likelihood $L(a;x)$ is $$L(a;x)= \begin{cases} a^n\prod_{i=1}^{n} x_i^{-2}&\text{if}\, \min(x_i) \geq a \\ 0&\text{otherwise} \end{cases}$$

I know that this case doesn't satisfy the regularity conditions for using the derivative to maximize the likelihood, so I'm not sure where to go with this problem.


You are correct that the problem doesn't satisfy the regularity conditions, but you can still find the MLE by looking at the derivative. Let's work with the log likelihood:

$$l(a;x) = n\log(a) -2\Sigma\log(x_i)$$

Clearly this is monotonically increasing in $a$ (this is all the looking at the derivative we're going to do.) So... our MLE $\hat{a}$ will be as large as possible. How large is that? We know $a \leq \min(x_i)$, so the largest $a$ can be is $\min(x_i)$. It follows that $\hat{a} = \min(x_i)$.


The answer by jbowman covers what you need to know to obtain the MLE, and I have nothing to add to that derivation. However, it is worth noting that in cases like this, where you have a sampling density that is monotonic in the parameter, it is usual for the maximum-likelihood estimator (MLE) to be bounded by the true parameter value and so you obtain a biased estimator. In this particular case you have $\hat{a} \geqslant a$ so the MLE will be biased upward. In cases like this it is useful to derive the distribution of the MLE and have a look at its expected value, to quantify the bias. You might also consider adjusting the estimator to correct the bias.

Letting $\hat{A} = \hat{A}(X_1, ..., X_n) = \min X_i$ be the estimator, and assuming for our derivation that $n>2$, we have the distribution function:

$$\begin{equation} \begin{aligned} F_{\hat{A}}(r) \equiv \mathbb{P}(\hat{A} \leqslant r) = \mathbb{P}(\min X_i \leqslant r) &= 1- \mathbb{P}(\min X_i > r) \\[6pt] &= 1-\mathbb{P}(X_1 > r) \cdot ... \cdot \mathbb{P}(X_n > r) \\[6pt] &= \begin{cases} 1-a^n r^{-n} & \text{ for } r \geqslant a, \\[6pt] 0 & \text{ for } r < a, \end{cases} \end{aligned} \end{equation}$$

and corresponding density function:

$$\begin{equation} \begin{aligned} f_{\hat{A}}(r) = \begin{cases} n a^n r^{-n-1} & \text{ for } r \geqslant a, \\[6pt] 0 & \text{ for } r < a. \end{cases} \end{aligned} \end{equation}$$

This gives the expected value:

$$\begin{equation} \begin{aligned} \mathbb{E}(\hat{A}) = \int \limits_a^\infty r f_{\hat{A}}(r) dr &= \int \limits_a^\infty n a^n r^{-n} dr \\[8pt] &= \Bigg[ \frac{n}{n-1} \cdot a^n r^{-(n-1)} \Bigg]_{r=a}^{r \rightarrow \infty} \\[8pt] &= \frac{n}{n-1} \cdot a. \end{aligned} \end{equation}$$

Correcting for bias: We can see from the above result that the MLE is biased upward, but it can easily be "corrected" by using the corresponding estimator:

$$A^* \equiv \frac{n-1}{n} \hat{A} = \frac{n-1}{n} \min X_i.$$

With some additional algebra it can be shown that the bias-adjusted MLE has moments:

$$\mathbb{E}(A^*) = a \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \mathbb{V}(A^*) = \frac{a^2}{n(n-2)}.$$

This latter estimator is unbiased and consistent. It is probably a preferable estimator to the MLE.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.