Making sense of this section on likelihood of order statistics

Question

I am currently studying the textbook In All Likelihood by Yudi Pawitan. In chapter 2 Elements of likelihood inference, the author presents the following example:

Example 2.4: Suppose $x$ is a sample form $N(\theta, 1)$; the likelihood of $\theta$ is $$L(\theta) = \phi(x - \theta) \equiv \dfrac{1}{\sqrt{2 \pi}} e^{-\frac{1}{2}(x - \theta)^2}.$$ The dashed curve in Figure 2.3(d) is the likelihood based on observing $x = 2.45$.

Suppose it is known only that $0.9 < x < 4$; then the likelihood of $\theta$ is $$L(\theta) = P(0.9 < X < 4) = \Phi(4 - \theta) - \Phi(0.9 - \theta),$$ where $\Phi(z)$ is the standard normal distribution function. The likelihood is shown in solid line in Figure 2.3(d).

Suppose $x_1, \dots, x_n$ are an identically and independently distributed (iid) sample from $N(\theta, 1)$, and only the maximum $x_{(n)}$ is reported, while the others are missing. The distribution function of $x_{(n)}$ is $$\begin{align} F(t) &= P(X_{(n)} \le t) \\ &= P(X_i \le t, \ \text{for each} \ i) \\ &= \{\Phi(t - \theta)\}^n . \end{align}$$ So, the likelihood based on observing $x_{(n)}$ is $$L(\theta) = p_{\theta}(x_{(n)}) = n \{ \Phi(x_{(n)} - \theta) \}^{n - 1} \phi(x_{(n)} - \theta).$$ Figure 2.3(d) shows this likelihood as a dotted line for $n = 5$ and $x_{(n)} = 3.5$.

There is a general heuristic to deal with order statistics for an iid sample from continuous density $p_\theta(x)$. Assume a finite precision $\epsilon$, and partition the real line into a regular grid of width $\epsilon$. Taking an iid sample $x_1, \dots, x_n$ is like performing a multinomial experiment: throw $n$ balls to cells with probability $p(x) \epsilon$ and record where they land. For example, the probability of the order statistics $x_{(1)}, \dots, x_{(n)}$ is approximately $$n! \epsilon^n \prod_i p_\theta(x_{(i)}).$$ Knowing only the maximum $x_{(n)}$, the multinomial argument yields immediately the likelihood given above. If only $x_{(1)}$ and $x_{(n)}$ are given, the likelihood of $\theta$ is $$L(\theta) = \dfrac{n(n - 1)}{2} \epsilon^2 p_\theta(x_{(1)})p_\theta(x_{(n)})\{ F_\theta(x_{(n)}) - F_\theta(x_{(1)}) \}^{n - 2},$$ where $F_\theta(x)$ is the underlying distribution function. $\square$

Everything until "There is a general heuristic to deal with order statistics ..." seems to make sense to me. However, after this, it isn't clear to me that what the author is saying is correct / makes sense. For instance, I don't see how knowing only the maximum $x_{(n)}$, the multinomial argument yields immediately the likelihood $L(\theta) = p_{\theta}(x_{(n)}) = n \{ \Phi(x_{(n)} - \theta) \}^{n - 1} \phi(x_{(n)} - \theta)$. Furthermore, I don't see how, if only $x_{(1)}$ and $x_{(n)}$ are given, the likelihood of $\theta$ is $L(\theta) = \dfrac{n(n - 1)}{2} \epsilon^2 p_\theta(x_{(1)})p_\theta(x_{(n)})\{ F_\theta(x_{(n)}) - F_\theta(x_{(1)}) \}^{n - 2}$. How does this part make sense?

Answer 1

The cumulative distribution of the maximum order statistic is

$$\begin{align} F(t) &= P(X_{(n)} \le t) \\ &= P(X_i \le t, \ \text{for each} \ i) \\ &= \{\Phi(t - \theta)\}^n . \end{align}$$

when you take the derivative you get

$$ f(t) = \frac{\partial F(t)}{\partial t} =\frac{\partial \{\Phi(t - \theta)\}^n}{\partial t} = n\phi(t - \theta)\{\Phi(t - \theta)\}^{n-1} $$

---

The heuristic of discretization

throw n balls to cells with probability $p(x)\epsilon$

is an alternative way to visualize this differentiation and sort of derive the density function directly.

The probability that the maximum is in the $k$-th bin is (approximately) equal to the probability that 1 ball is in the $k$-th bin multiplied with the probability that $n-1$ others are in the $k$-th bin or lower.

$$\quad L(\theta) = \underbrace{n}_{\substack{\llap{\text{multiplicati}} \rlap{\text{on factor}} \\ \llap{\text{for number of p}} \rlap{\text{ermutations}}\\ \llap{\text{a ball can be th}} \rlap{\text{e maximum}} } } \cdot \overbrace{\epsilon p_\theta(x_{(n)})}^{\llap{\text{probability a b}}\rlap{\text{all in bin $x_{(n)}$}}} \cdot \underbrace{\{ F_\theta(x_{(n)}) \}^{n - 1}}_{\substack{\text{probability $n-1$ balls}\\\text{in bin $x_{(n)}$ or lower}}}$$

You get a multiplicity of $n$ because there are $n$ ways that a ball is the maximum.*

For the other case

$$\quad L(\theta) = \underbrace{\frac{(n-1)n}{2}}_{\substack{\llap{\text{multiplication fac}} \rlap{\text{tor for the}} \\ \llap{\text{number of ways to s}} \rlap{\text{elect two balls}}\\ \llap{\text{for the minimum an}} \rlap{\text{d maximum}} } } \cdot \overbrace{\epsilon^2 p_\theta(x_{(1)}) p_\theta(x_{(n)})}^{\substack{\llap{\text{probability b}}\rlap{\text{all in bin $x_{(1)}$}} \\ \llap{\text{and a bal}}\rlap{\text{l in bin $x_{(n)}$}}}} \cdot \underbrace{\{ F_\theta(x_{(n)}) -F_\theta(x_{(1)}) \}^{n - 2}}_{\substack{\text{probability $n-2$ balls}\\\text{inbetween bin $x_{(1)}$ and $x_{(n)}$}}}$$

---

*^{The formula is not entirely correct and there is some double counting. For example say we roll 2 dice, then the probability that six is the highest roll is not $2\cdot (1/6) \cdot (6/6) = 2/6$. This formula is the sum of the probabilities for the case '1st roll is 6 and the 2nd roll is 6 or lower' and the case '2nd roll is 6 and the 1st roll is 6 or lower'. The situation of double sixes is counted twice in this formula. The true result is $11/36$. The difference is $1/36$ which is $\epsilon^2$. When we take the limit $\epsilon \to 0$ then this error approaches zero.}