# MLE for the maximum of n values that are observed only with noise

Suppose $$x_1, ..., x_n$$ is a fixed set of real numbers. Let $$\epsilon_1, ..., \epsilon_n \sim N(0, \sigma^2)$$ be i.i.d. with known $$\sigma^2$$, and suppose we get to observe only $$z_i = x_i + \epsilon_i$$, $$i=1, ..., n$$. What is the MLE of $$\max(x_1, ..., x_n)$$?

• Intuitively, it should be $\max(z_1,...,z_n).$ The question is how to show this. I'm working on an answer now.
– dlnB
Commented Apr 29, 2020 at 17:51
• @dlnB is right; it follows from invariance of MLE. stats.stackexchange.com/q/459605/119261 Commented Apr 29, 2020 at 18:13
• Got it! See my answer below.
– dlnB
Commented Apr 29, 2020 at 18:15

It is easiest to think of $$(x_1,...,x_n)$$ as parameters and $$(z_1,...,z_n)$$ as data. Then the joint likelihood function is $$L(x_1,...,x_n|z_1,...,z_n) = \prod_{i=1}^n \phi(\frac{z_i-x_i}{\sigma}),$$ and joint log-likelihood function is $$\ell(x_1,...,x_n|z_1,...,z_n) = \sum_{i=1}^n \ln \phi(\frac{z_i-x_i}{\sigma}),$$ Solving the first-order conditions gives MLE estimator $$\hat{x_i}=z_i$$ for $$i=1,...,n$$.

The invariance property of MLE says if $$\hat{\theta}$$ is the MLE estimator of $$\theta$$, then for any $$f(\theta)$$, the MLE estimator is $$f(\hat{\theta})$$. It follows that the MLE estimator of $$\max(x_1,...,x_n)$$ is $$\max(\hat{x_1},...,\hat{x_n}) = \max(z_1,...,z_n).$$

• Interesting. I guess this could become intuitive if I were to understand the invariance property. Anyhow this is an impressively biased MLE when n > 1. Commented Apr 29, 2020 at 18:21
• Very interesting that this is biased. How did you find this?
– dlnB
Commented Apr 29, 2020 at 18:23
• Consider the edge case where all the x's are 0. $E(max(z_1, ..., z_n))$ is then just the expectation of the maximum of n Gaussians, which is strictly positive for n > 1: math.stackexchange.com/a/89037/262048 . This does not contradict your answer though Commented Apr 29, 2020 at 18:27
• Clever. Nice observation.
– dlnB
Commented Apr 29, 2020 at 18:30
• The proof is simpler when the function $\tau(\cdot)$ is 1-1, but the theorem holds for any function $\tau(\cdot)$. See Casella (1990): Theorem 7.2.1 (Invariance Property of Maximum Likelihood Estimators): If $\hat{\theta}$ is the MLE of $\theta$, then for any function $\tau (\theta)$, the MLE of $\tau(\theta)$ is $\tau(\hat{\theta})$.
– dlnB
Commented Apr 29, 2020 at 19:43