5
$\begingroup$

Suppose $x_1, ..., x_n$ is a fixed set of real numbers. Let $\epsilon_1, ..., \epsilon_n \sim N(0, \sigma^2)$ be i.i.d. with known $\sigma^2$, and suppose we get to observe only $z_i = x_i + \epsilon_i$, $i=1, ..., n$. What is the MLE of $\max(x_1, ..., x_n)$?

$\endgroup$
3
  • $\begingroup$ Intuitively, it should be $\max(z_1,...,z_n).$ The question is how to show this. I'm working on an answer now. $\endgroup$
    – dlnB
    Apr 29, 2020 at 17:51
  • $\begingroup$ @dlnB is right; it follows from invariance of MLE. stats.stackexchange.com/q/459605/119261 $\endgroup$ Apr 29, 2020 at 18:13
  • $\begingroup$ Got it! See my answer below. $\endgroup$
    – dlnB
    Apr 29, 2020 at 18:15

1 Answer 1

5
$\begingroup$

It is easiest to think of $(x_1,...,x_n)$ as parameters and $(z_1,...,z_n)$ as data. Then the joint likelihood function is $$L(x_1,...,x_n|z_1,...,z_n) = \prod_{i=1}^n \phi(\frac{z_i-x_i}{\sigma}),$$ and joint log-likelihood function is $$\ell(x_1,...,x_n|z_1,...,z_n) = \sum_{i=1}^n \ln \phi(\frac{z_i-x_i}{\sigma}),$$ Solving the first-order conditions gives MLE estimator $\hat{x_i}=z_i$ for $i=1,...,n$.

The invariance property of MLE says if $\hat{\theta}$ is the MLE estimator of $\theta$, then for any $f(\theta)$, the MLE estimator is $f(\hat{\theta})$. It follows that the MLE estimator of $\max(x_1,...,x_n)$ is $\max(\hat{x_1},...,\hat{x_n}) = \max(z_1,...,z_n).$

$\endgroup$
8
  • $\begingroup$ Interesting. I guess this could become intuitive if I were to understand the invariance property. Anyhow this is an impressively biased MLE when n > 1. $\endgroup$
    – zkurtz
    Apr 29, 2020 at 18:21
  • $\begingroup$ Very interesting that this is biased. How did you find this? $\endgroup$
    – dlnB
    Apr 29, 2020 at 18:23
  • $\begingroup$ Consider the edge case where all the x's are 0. $E(max(z_1, ..., z_n))$ is then just the expectation of the maximum of n Gaussians, which is strictly positive for n > 1: math.stackexchange.com/a/89037/262048 . This does not contradict your answer though $\endgroup$
    – zkurtz
    Apr 29, 2020 at 18:27
  • $\begingroup$ Clever. Nice observation. $\endgroup$
    – dlnB
    Apr 29, 2020 at 18:30
  • $\begingroup$ The proof is simpler when the function $\tau(\cdot)$ is 1-1, but the theorem holds for any function $\tau(\cdot)$. See Casella (1990): Theorem 7.2.1 (Invariance Property of Maximum Likelihood Estimators): If $\hat{\theta}$ is the MLE of $\theta$, then for any function $\tau (\theta)$, the MLE of $\tau(\theta)$ is $\tau(\hat{\theta})$. $\endgroup$
    – dlnB
    Apr 29, 2020 at 19:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.