# Making sense of this section on likelihood of order statistics

I am currently studying the textbook In All Likelihood by Yudi Pawitan. In chapter 2 Elements of likelihood inference, the author presents the following example:

Example 2.4: Suppose $$x$$ is a sample form $$N(\theta, 1)$$; the likelihood of $$\theta$$ is $$L(\theta) = \phi(x - \theta) \equiv \dfrac{1}{\sqrt{2 \pi}} e^{-\frac{1}{2}(x - \theta)^2}.$$ The dashed curve in Figure 2.3(d) is the likelihood based on observing $$x = 2.45$$.

Suppose it is known only that $$0.9 < x < 4$$; then the likelihood of $$\theta$$ is $$L(\theta) = P(0.9 < X < 4) = \Phi(4 - \theta) - \Phi(0.9 - \theta),$$ where $$\Phi(z)$$ is the standard normal distribution function. The likelihood is shown in solid line in Figure 2.3(d).

Suppose $$x_1, \dots, x_n$$ are an identically and independently distributed (iid) sample from $$N(\theta, 1)$$, and only the maximum $$x_{(n)}$$ is reported, while the others are missing. The distribution function of $$x_{(n)}$$ is \begin{align} F(t) &= P(X_{(n)} \le t) \\ &= P(X_i \le t, \ \text{for each} \ i) \\ &= \{\Phi(t - \theta)\}^n . \end{align} So, the likelihood based on observing $$x_{(n)}$$ is $$L(\theta) = p_{\theta}(x_{(n)}) = n \{ \Phi(x_{(n)} - \theta) \}^{n - 1} \phi(x_{(n)} - \theta).$$ Figure 2.3(d) shows this likelihood as a dotted line for $$n = 5$$ and $$x_{(n)} = 3.5$$.

There is a general heuristic to deal with order statistics for an iid sample from continuous density $$p_\theta(x)$$. Assume a finite precision $$\epsilon$$, and partition the real line into a regular grid of width $$\epsilon$$. Taking an iid sample $$x_1, \dots, x_n$$ is like performing a multinomial experiment: throw $$n$$ balls to cells with probability $$p(x) \epsilon$$ and record where they land. For example, the probability of the order statistics $$x_{(1)}, \dots, x_{(n)}$$ is approximately $$n! \epsilon^n \prod_i p_\theta(x_{(i)}).$$ Knowing only the maximum $$x_{(n)}$$, the multinomial argument yields immediately the likelihood given above. If only $$x_{(1)}$$ and $$x_{(n)}$$ are given, the likelihood of $$\theta$$ is $$L(\theta) = \dfrac{n(n - 1)}{2} \epsilon^2 p_\theta(x_{(1)})p_\theta(x_{(n)})\{ F_\theta(x_{(n)}) - F_\theta(x_{(1)}) \}^{n - 2},$$ where $$F_\theta(x)$$ is the underlying distribution function. $$\square$$

Everything until "There is a general heuristic to deal with order statistics ..." seems to make sense to me. However, after this, it isn't clear to me that what the author is saying is correct / makes sense. For instance, I don't see how knowing only the maximum $$x_{(n)}$$, the multinomial argument yields immediately the likelihood $$L(\theta) = p_{\theta}(x_{(n)}) = n \{ \Phi(x_{(n)} - \theta) \}^{n - 1} \phi(x_{(n)} - \theta)$$. Furthermore, I don't see how, if only $$x_{(1)}$$ and $$x_{(n)}$$ are given, the likelihood of $$\theta$$ is $$L(\theta) = \dfrac{n(n - 1)}{2} \epsilon^2 p_\theta(x_{(1)})p_\theta(x_{(n)})\{ F_\theta(x_{(n)}) - F_\theta(x_{(1)}) \}^{n - 2}$$. How does this part make sense?

• The multinomial argument isn't needed for the maximum example: just differentiate the distribution function to obtain the result. The general argument is similar to, and in the same spirit, as the one I made in a post at stats.stackexchange.com/a/225990/919.
– whuber
Dec 3, 2022 at 22:07
• @whuber I don't see how this addresses my questions. Dec 4, 2022 at 8:38
• See, then, the first section of stats.stackexchange.com/a/130174/919 for more details.
– whuber
Dec 7, 2022 at 16:00

The cumulative distribution of the maximum order statistic is

\begin{align} F(t) &= P(X_{(n)} \le t) \\ &= P(X_i \le t, \ \text{for each} \ i) \\ &= \{\Phi(t - \theta)\}^n . \end{align}

when you take the derivative you get

$$f(t) = \frac{\partial F(t)}{\partial t} =\frac{\partial \{\Phi(t - \theta)\}^n}{\partial t} = n\phi(t - \theta)\{\Phi(t - \theta)\}^{n-1}$$

The heuristic of discretization

throw n balls to cells with probability $$p(x)\epsilon$$

is an alternative way to visualize this differentiation and sort of derive the density function directly.

The probability that the maximum is in the $$k$$-th bin is (approximately) equal to the probability that 1 ball is in the $$k$$-th bin multiplied with the probability that $$n-1$$ others are in the $$k$$-th bin or lower.

$$\quad L(\theta) = \underbrace{n}_{\substack{\llap{\text{multiplicati}} \rlap{\text{on factor}} \\ \llap{\text{for number of p}} \rlap{\text{ermutations}}\\ \llap{\text{a ball can be th}} \rlap{\text{e maximum}} } } \cdot \overbrace{\epsilon p_\theta(x_{(n)})}^{\llap{\text{probability a b}}\rlap{\text{all in bin x_{(n)}}}} \cdot \underbrace{\{ F_\theta(x_{(n)}) \}^{n - 1}}_{\substack{\text{probability n-1 balls}\\\text{in bin x_{(n)} or lower}}}$$

You get a multiplicity of $$n$$ because there are $$n$$ ways that a ball is the maximum.*

For the other case

$$\quad L(\theta) = \underbrace{\frac{(n-1)n}{2}}_{\substack{\llap{\text{multiplication fac}} \rlap{\text{tor for the}} \\ \llap{\text{number of ways to s}} \rlap{\text{elect two balls}}\\ \llap{\text{for the minimum an}} \rlap{\text{d maximum}} } } \cdot \overbrace{\epsilon^2 p_\theta(x_{(1)}) p_\theta(x_{(n)})}^{\substack{\llap{\text{probability b}}\rlap{\text{all in bin x_{(1)}}} \\ \llap{\text{and a bal}}\rlap{\text{l in bin x_{(n)}}}}} \cdot \underbrace{\{ F_\theta(x_{(n)}) -F_\theta(x_{(1)}) \}^{n - 2}}_{\substack{\text{probability n-2 balls}\\\text{inbetween bin x_{(1)} and x_{(n)}}}}$$

*The formula is not entirely correct and there is some double counting. For example say we roll 2 dice, then the probability that six is the highest roll is not $$2\cdot (1/6) \cdot (6/6) = 2/6$$. This formula is the sum of the probabilities for the case '1st roll is 6 and the 2nd roll is 6 or lower' and the case '2nd roll is 6 and the 1st roll is 6 or lower'. The situation of double sixes is counted twice in this formula. The true result is $$11/36$$. The difference is $$1/36$$ which is $$\epsilon^2$$. When we take the limit $$\epsilon \to 0$$ then this error approaches zero.

• Hmm, ok, I think that makes sense for $L(\theta) = p_{\theta}(x_{(n)}) = n \{ \Phi(x_{(n)} - \theta) \}^{n - 1} \phi(x_{(n)} - \theta)$. But for $L(\theta) = \dfrac{n(n - 1)}{2} \epsilon^2 p_\theta(x_{(1)})p_\theta(x_{(n)})\{ F_\theta(x_{(n)}) - F_\theta(x_{(1)}) \}^{n - 2}$, where does the $\dfrac{1}{2}$ come from? Dec 7, 2022 at 4:28
• It comes from $\binom{n}{2}.$
– whuber
Dec 7, 2022 at 4:40
• @ThePointer $$\frac{n!}{2!(n-2)!} = \frac{n(n-1)}{2}$$ it is the different number of ways that you can pick from $n$ non-identical balls, two balls that becomes the minimum or the maximum. For example with four balls you can have (1,2) (1,3) (1,4) (2,3) (2,4) (3,4) different ways. You could alternatively also see this as a sum $\sum_{k=1}^{n-1} k$, ie $3+2+1 = 6 = \frac{(4-1)4}{2}$.... Dec 7, 2022 at 9:32
• The idea behind the binomial coefficient is that $n!$ is the total number of ways that you can permute $n$ balls and $2!$ and $(n-2)!$ are the number of ways that you can permute groups of size 2 and (n-2) which is how often the same result occurs but in a different order. Dec 7, 2022 at 9:36
• I think there is a typo in the exponent in the last term for the min/max case. Dec 11, 2022 at 23:42