22
$\begingroup$

Let $\mathbf{X}= (x_1, x_2, \dots x_n)$ be a random sample from the uniform distribution on $(a,b)$, where $a < b$. Let $Y_1$ and $Y_n$ be the largest and smallest order statistics. Show that the statistic $(Y_1, Y_n)$ is a jointly complete sufficient statistic for the parameter $\theta = (a, b)$.

It is no problem for me to show sufficiency using factorization.

Question: How do I show completeness? Preferably I would like a hint.

Attempt: I can show $\mathbb E[g(T(x))] = 0$ implies $g(T(x)) = 0$ for the one parameter uniform distribution, but I am getting stuck on the two parameter uniform distribution.

I tried playing around with $\mathbb E[g(Y_1, Y_n)]$ and using the joint distribution of $Y_1$ and $Y_n$, but I am not sure if I am going in the correct direction, as the calculus is tripping me up.

$\endgroup$
1
  • 2
    $\begingroup$ Please add the [self-study] tag & read its wiki. Note that you can use Latex formatting for math by putting dollars around, e.g. $x$ produces $x$. I have tried to typeset some of your maths but feel free to change or revert if you are not happy with the outcome. You might prefer the notation $\vec x$ for $\vec x$ instead of $\mathbf x$ for $\mathbf x$. $\endgroup$
    – Silverfish
    Jul 25, 2016 at 19:25

3 Answers 3

11
$\begingroup$

Let's take care of the routine calculus for you, so you can get to the heart of the problem and enjoy formulating a solution. It comes down to constructing rectangles as unions and differences of triangles.

First, choose values of $a$ and $b$ that make the details as simple as possible. I like $a=0,b=1$: the univariate density of any component of $X=(X_1,X_2,\ldots,X_n)$ is just the indicator function of the interval $[0,1]$.

Let's find the distribution function $F$ of $(Y_1,Y_n)$. By definition, for any real numbers $y_1 \le y_n$ this is

$$F(y_1,y_n) = \Pr(Y_1\le y_1\text{ and } Y_n \le y_n).\tag{1}$$

The values of $F$ are obviously $0$ or $1$ in case any of $y_1$ or $y_n$ is outside the interval $[a,b] = [0,1]$, so let's assume they're both in this interval. (Let's also assume $n\ge 2$ to avoid discussing trivialities.) In this case the event $(1)$ can be described in terms of the original variables $X=(X_1,X_2,\ldots,X_n)$ as "at least one of the $X_i$ is less than or equal to $y_1$ and none of the $X_i$ exceed $y_n$." Equivalently, all the $X_i$ lie in $[0,y_n]$ but it is not the case that all of them lie in $(y_1,y_n]$.

Because the $X_i$ are independent, their probabilities multiply and give $(y_n-0)^n = y_n^n$ and $(y_n-y_1)^n$, respectively, for these two events just mentioned. Thus,

$$F(y_1,y_n) = y_n^n - (y_n-y_1)^n.$$

The density $f$ is the mixed partial derivative of $F$,

$$f(y_1,y_n) = \frac{\partial^2 F}{\partial y_1 \partial y_n}(y_1,y_n) = n(n-1)(y_n-y_1)^{n-2}.$$

The general case for $(a,b)$ scales the variables by the factor $b-a$ and shifts the location by $a$. Thus, for $a \lt y_1 \le y_n \lt b$,

$$F(y_1,y_n; a,b) = \left(\left(\frac{y_n-a}{b-a}\right)^n - \left(\frac{y_n-a}{b-a} - \frac{y_1-a}{b-a}\right)^n\right) = \frac{(y_n-a)^n - (y_n-y_1)^n}{(b-a)^n}.$$

Differentiating as before we obtain

$$f(y_1,y_n; a,b) = \frac{n(n-1)}{(b-a)^n}(y_n-y_1)^{n-2}.$$

Consider the definition of completeness. Let $g$ be any measurable function of two real variables. By definition,

$$\eqalign{E[g(Y_1,Y_n)] &= \int_{y_1}^b\int_a^b g(y_1,y_n) f(y_1,y_n)dy_1dy_n\\ &\propto\int_{y_1}^b\int_a^b g(y_1,y_n) (y_n-y_1)^{n-2} dy_1dy_n.\tag{2} }$$

We need to show that when this expectation is zero for all $(a,b)$, then it's certain that $g=0$ for any $(a,b)$.

Here's your hint. Let $h:\mathbb{R}^2\to \mathbb{R}$ be any measurable function. I would like to express it in the form suggested by $(2)$ as $h(x,y)=g(x,y)(y-x)^{n-2}$. To do that, obviously we must divide $h$ by $(y-x)^{n-2}$. Unfortunately, for $n\gt 2$ this isn't defined whenever $y=x$. The key is that this set has measure zero so we can neglect it.

Accordingly, given any measurable $h$, define

$$g(x,y) = \left\{\matrix{h(x,y)/(y-x)^{n-2} & x \ne y \\ 0 & x=y}\right.$$

Then $(2)$ becomes

$$\int_{y_1}^b\int_a^b h(y_1,y_n) dy_1dy_n \propto E[g(Y_1,Y_n)].\tag{3}$$

(When the task is showing that something is zero, we may ignore nonzero constants of proportionality. Here, I have dropped $n(n-1)/(b-a)^{n-2}$ from the left hand side.)

This is an integral over a right triangle with hypotenuse extending from $(a,a)$ to $(b,b)$ and vertex at $(a,b)$. Let's denote such a triangle $\Delta(a,b)$.

Ergo, what you need to show is that if the integral of an arbitrary measurable function $h$ over all triangles $\Delta(a,b)$ is zero, then for any $a\lt b$, $h(x,y)=0$ (almost surely) for all $(x,y)\in \Delta(a,b)$.

Although it might seem we haven't gotten any further, consider any rectangle $[u_1,u_2]\times [v_1,v_2]$ wholly contained in the half-plane $y \gt x$. It can be expressed in terms of triangles:

$$[u_1,u_2]\times [v_1,v_2] = \Delta(u_1,v_2) \setminus\left(\Delta(u_1,v_1) \cup \Delta(u_2,v_2)\right)\cup \Delta(u_2,v_1).$$

Figure showing the three triangles overlapping to produce the rectangle

In this figure, the rectangle is what is left over from the big triangle when we remove the overlapping red and green triangles (which double counts their brown intersection) and then replace their intersection.

Consequently, you may immediately deduce that the integral of $h$ over all such rectangles is zero. It remains only to show that $h(x,y)$ must be zero (apart from its values on some set of measure zero) whenever $y \gt x$. The proof of this (intuitively clear) assertion depends on what approach you want to take to the definition of integration.

$\endgroup$
4
  • $\begingroup$ I tried to set equation 3 equal to zero, take the derivative on both sides and interchange the signs (a reflex action I guess) but the results look quite scary [1]. Is there a more reasonable approach? [1] en.wikipedia.org/wiki/Leibniz_integral_rule#Higher_dimensions $\endgroup$
    – mugen
    Feb 19, 2019 at 21:33
  • 1
    $\begingroup$ Consider finite collections of smaller and smaller triangles all lying along the hypotenuse in the picture and take the limit as the diameter of the largest triangle in the collection goes to zero. $\endgroup$
    – whuber
    Feb 19, 2019 at 22:52
  • $\begingroup$ @whuber, I hope you wanted to write $y=x$ rather than $y-x.$ But if it is not, please feel free to rollback. $\endgroup$ Feb 28, 2023 at 4:13
  • 1
    $\begingroup$ @User1865345 Thank you for catching that! $\endgroup$
    – whuber
    Feb 28, 2023 at 13:56
1
$\begingroup$

Following @whuber's answer, we have the joint density of $(Y_{(1)},Y_{(n)})$ $$f(y_1, y_n,a,b)=\frac{n(n-1)}{(b-a)^n}(y_n-y_1)^{n-1},\quad\forall a\leq y_1\leq y_n\leq b\text{ and }a,b\in\mathbb{R}$$

For any function $g(x_1,x_n)$ so that $\mathbb{E}_{a,b}\left[g(y_1,y_n)\right]=0,\forall a,b\in\mathbb{R}\text{ and }a<b$, we have $$0=\frac{n(n-1)}{(b-a)^n}\int_{a}^{b}\int_{a}^{y_n}g(y_1,y_n)(y_n-y_1)^{n-2} dy_1dy_n,\forall a,b\in\mathbb{R}\text{ and }a<b.$$ The integral area of $(y_1, y_n)$ is a triangle with vertices $(a,a)$, $(a,b)$ and $(b,b)$. With varying $a\in\mathbb{R}$ $b\in\mathbb{R}$ $a<b$, these triangles generate the Borel $\sigma$-algebra of $\mathcal{B}=\{(x,z)\in\mathbb{R}^2:x\leq z\}$. Thus $$0=\frac{n(n-1)}{(b-a)^n}\int_{A}g(y_1,y_n)(y_n-y_1)^{n-2} d(y_1,y_n),\text{ for any Borel set }A\subset\mathcal{B}.$$ This means $$g(y_1,y_n)(y_n-y_1)^{n-2}\equiv0,a.e.\iff g\equiv 0,a.e.$$ Thus we conclude $(Y_{(1)},Y_{(n)})$ is a complete statistic for $(a,b).$

$\endgroup$
1
$\begingroup$

I think it is worth elaborating a little bit the step of how to reach $g = 0$ a.e. $\lambda$ from the integral equation in @Tan's and @whuber's answer, as it demonstrates some classical measure theory techniques that could be used in other similar completeness proving problems.

Denote $g(x, y)(y - x)^{n - 2}$ by $f(x, y)$, then Tan's answer showed that \begin{align} \int_a^b\int_a^y f(x, y)dxdy = 0\; \text{ for all } a < b. \tag{1} \end{align}

whuber's clever geometric argument shows that $(1)$ implies that \begin{align} \int_I f(x, y)dxdy = 0\; \text{ for all } I \in \mathscr{I}. \tag{2} \end{align} where $\mathscr{I}$ is the class of bounded rectangles in $\mathbb{R}^2$:
\begin{align} \mathscr{I} = \{(a_1, b_1] \times (a_2, b_2]: a_1 < b_1, a_2 < b_2\}. \end{align}

Our goal is to prove by that $f = 0$ a.e. on $\mathbb{R}^2$ under the condition $(2)$ (which obviously implies $g = 0$ a.e. on $\mathbb{R}^2$). This is implied$^\dagger$ by $f = 0$ a.e. on $I_M := (-M, M] \times (-M, M]$ for arbitrary $M > 0$, to which we prove it below.

It is well known that $\mathscr{I}$ generates the Borel $\sigma$-field $\mathscr{R}^2$ on $\mathbb{R}^2$, hence $\mathscr{I} \cap I_M$ generates the $\sigma$-field $\mathscr{R}^2 \cap I_M := \{A \cap I_M: A \in \mathscr{R}^2\}$ in $I_M$ (see Theorem 10.1 in Probability and Measure by Patrick Billingsley for the proof). Hence if we can show that the class \begin{align} \mathscr{A} := \left\{A \in \mathscr{R}^2 \cap I_M: \int_A f(x, y)dxdy = 0\right\} \tag{3} \end{align} is a $\sigma$-field in $I_M$, then $\mathscr{I} \cap I_M \subset \mathscr{A}$ (which is implied by $(2)$) and $\sigma(\mathscr{I} \cap I_M) = \mathscr{R}^2 \cap I_M$ together imply that $\mathscr{R}^2 \cap I_M \subset \mathscr{A}$. Since $f$ is measurable, $A_1 := \{(x, y): f(x, y) > 0\} \in \mathscr{R}^2$ and $A_2 := \{(x, y): f(x, y) < 0\} \in \mathscr{R}^2$, it then follows by $\mathscr{R}^2 \cap I_M \subset \mathscr{A}$ that $A_1 \cap I_M \in \mathscr{A}$, $A_2 \cap I_M \in \mathscr{A}$. Therefore $(3)$ implies that \begin{align} \int_{A_1 \cap I_M}f(x, y)dxdy = 0, \quad \int_{A_2 \cap I_M} -f(x, y)dxdy = 0. \tag{4} \end{align}

Note that $fI_{A_1} = f^+ = \max(f, 0) \geq 0$ and $-fI_{A_2} = f^- = \max(-f, 0) \geq 0$, $(4)$ can be rewritten as \begin{align} \int_{I_M}f^+(x, y)dxdy = \int_{I_M}f^-(x, y)dxdy = 0, \end{align} which implies that $f^+ = f^- = 0$ a.e. on $I_M$. Therefore, $f = f^+ - f^- = 0$ a.e. on $I_M$.

Therefore, to complete the proof, it remains to show that $\mathscr{A}$ is a $\sigma$-field. In fact, because $\mathscr{I} \cap I_M$ is a $\pi$-system, it suffices to show that $\mathscr{A}$ is a $\lambda$-system in view of Dynkin's $\pi$-$\lambda$ theorem. To this end:

  1. $I_M \in \mathscr{A}$. This follows from $(2)$ directly.
  2. If $A \in \mathscr{A}$, then \begin{align} \int_{I_M - A}f(x)dxdy = \int_{I_M}f(x)dxdy - \int_A f(x)dxdy = 0 - 0 = 0, \end{align} which shows $I_M - A$ lies in $\mathscr{A}$. Therefore $\mathscr{A}$ is closed under the formation of complementation.
  3. If $A_n$ lies in $\mathscr{A}$ for all $n$ and $A_1, A_2, \ldots$ are disjoint, then $|fI_{\cup_n A_n}| \leq |f|$ (note that $\int_{I_M}fd\lambda = 0$ implies that $f$ is integrable in $I_M$) and Lebesgue's dominated convergence theorem imply that \begin{align} \int_{\cup_n A_n} fd\lambda = \sum_n \int_{A_n}fd\lambda = 0, \end{align} which shows $\cup_n A_n$ lies in $\mathscr{A}$. Therefore $\mathscr{A}$ is closed under the formation of countable disjoint unions.

This completes the proof.


$^\dagger$: For each $m \in \mathbb{N}$, define $B_m = \{x \in I_m: f(x) \neq 0\}$. Then the sequence $\{B_m\}$ is increasing and converges to the set $\cup_{m = 1}^\infty B_m = \{x \in \mathbb{R}^2: f(x) \neq 0\}$. It then follows by the continuity from below of $\lambda$ and $f = 0$ a.e. on every $I_m$ that $\lambda(\cup_{m = 1}^\infty B_m) = \lim_{m \to \infty}\lambda(B_m) = 0$, i.e., $f = 0$ a.e. on $\mathbb{R}^2$.

$\endgroup$
6
  • $\begingroup$ It has been a quick read for me (I have engrossed on this of late on a related question). $+1$ for while it is not painstakingly detailed, it mentions the arguments what the authors tend to sideline assuming we are already aware. $\endgroup$ Feb 28, 2023 at 21:39
  • 1
    $\begingroup$ @User1865345 Coincidentally, it is your answer to this question, which I upvoted, to inspire me to elaborate the "sideline" step in such problems. As I mentioned in my answer, there seems still an additional assumption that is not in the definition of completeness is needed to connect every dots. I am really interested in to see if any publications discussed it. $\endgroup$
    – Zhanxiong
    Feb 28, 2023 at 21:43
  • $\begingroup$ Thanks for the upvote (although it gained a downvote and I am not sure about the reason ¯_(ツ)_/¯). But point is I have gone through quite a few literatures and papers but I have felt that either they are assuming something happening behind the scenes or altogether leave the argument. These are standard routine measure theoretic exercises. But I would very much appreciate a comprehensive paper detailing a canonical post like this. $\endgroup$ Feb 28, 2023 at 21:49
  • 1
    $\begingroup$ @User1865345 Agree. Most papers/textbooks I am aware of directly generalizes $(2)$ to $\int_A f(x, y)dxdy = 0$ for all Borel sets $A$. While $\mathscr{I}$ indeed generates the Borel $\sigma$-field $\mathscr{R}^2$, I am not sure how the loop is closed without showing $\mathscr{A}$ itself is a $\sigma$-field or $\lambda$-system (which probably needs more than measurability of $f$). I very welcome the discussion on this technicality. $\endgroup$
    – Zhanxiong
    Feb 28, 2023 at 21:55
  • 1
    $\begingroup$ I slowed my pace reading between the lines to comprehend the philosophy which is ingenious: show $f=0~\textrm{a.s.}$ on $I_M:$ this is a simple manifestation of Dynkin's theorem. Finally show it implies $f=0~\textrm{a.s.}$ on $\mathbb R^2.$ Yes, this post is now looking better and insightful. $\endgroup$ Mar 1, 2023 at 13:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.