1
$\begingroup$

Problem

Show that the maximum of $x_1,...,x_n \sim \mathrm{Uniform}(0,\theta)$ is a sufficient statistic for $\theta$.

Background

This question has been asked before, but most answers tackle the problem with the Factorization Theorem. I am trying to understand the definition of sufficiency given in Wasserman's All of Statistics, so I've been trying to solve this problem without appealing to theorems.

Here is the definition of a sufficient statistic given by Wasserman:

Denote IID $x_1,...,x_n \sim F$ as $x^n$. Write $x^n \leftrightarrow y^n$ if $f(x^n;\theta) = cf(y^n; \theta)$ for some constant $c$ that might depend on $x^n$ and $y^n$ but not on $\theta$. A statistic $T(x^n)$ is sufficient if $T(x^n) \leftrightarrow T(y^n)$ implies that $x^n \leftrightarrow y^n$.

As I wrote up this question, I think I figured out the problem, but I would greatly appreciate if someone can check my proof.

Attempt

The joint PDF of IID $x_1,...,x_n \sim \mathrm{Uniform(0,\theta)}$ can be written as

$$ \begin{align*} f(x^n;\theta) &= \Pi_{i=1}^n \theta^{-1} \mathbf{1}_{[0,\theta]}(x_i)\\ &= \theta^{-n}\mathbf{1}_{[0,\infty]}(\min x^n)\mathbf{1}_{(-\infty,\theta]}(\max x^n) \end{align*} $$

Note: Given $x^n = x_1,...,x_n$, $T(x^n) = \max(x^n)$ is a single number, so $\max T(x^n) = T(x^n)$. Now suppose that $T(x^n) \leftrightarrow T(y^n)$. Then, by definition,

$$f(T(x^n);\theta) = cf(T(y^n); \theta)$$

for some constant $c$ that might depend on $x^n,y^n$.

This implies

$$ \begin{align*} f(x^n;\theta) &= \theta^{-n}\mathbf{1}_{[0,\infty]}(\min x^n)\mathbf{1}_{(-\infty,\theta]}(\max x^n)\\ &= \theta^{-n}\mathbf{1}_{[0,\infty]}(\min T(x^n))\mathbf{1}_{(-\infty,\theta]}(\max T(x^n))\\ &= f(T(x^n);\theta)\\ &= cf(T(y^n);\theta)\\ &= c\theta^{-n}\mathbf{1}_{[0,\infty]}(\min T(y^n))\mathbf{1}_{(-\infty,\theta]}(\max T(y^n))\\ &= c\theta^{-n}\mathbf{1}_{[0,\infty]}(\min y^n)\mathbf{1}_{(-\infty,\theta]}(\max y^n)\\ &= cf(y^n;\theta) \end{align*} $$

Thus $x^n \leftrightarrow y^n$, and we conclude that $T(x^n)$ is a sufficient statistic for $\theta$.

Edit

My question is more about correctly intepreting Wasserman's definition rather than the specific problem at hand. The problem is just one way to flesh-out my understanding of the definition.

$\endgroup$
6
  • $\begingroup$ No, although it seems related. I actually tried answering my own question in my write-up, so I'm hoping someone can confirm that I've interpreted the definition correctly. $\endgroup$ – nwsteg Mar 26 at 21:18
  • $\begingroup$ I'm not trying to prove that $\max x^n$ is minimal sufficient, and I'm not trying to prove it using the theorem in your linked question. $\endgroup$ – nwsteg Mar 26 at 21:22
  • 4
    $\begingroup$ (i) This definition is terrible. (ii) The note about $\max T(x^n) = T(x^n)$ does not make sense. (iii) It connects with the fact that you appear to miss that the distribution of $T(x^n)$ need be constructed, rather than being the density of $x^n$ at $T(x^n)$ as several lines in your equations hint. $\endgroup$ – Xi'an Mar 27 at 8:13
  • 1
    $\begingroup$ Yes, you're correct -- I thought I was supposed to plug T into the PDF for X. Now I see that I need to construct the distribution of T. $\endgroup$ – nwsteg Mar 27 at 16:19
4
$\begingroup$

If one deconstructs Larry's (rather awful!) definition

Denote iid $X_1,...,X_n\sim F$ as ${X}^n$. Write $x^n↔y^n$ if $f(x^n;θ)=cf(y^n;θ)$ for some constant $c$ that might depend on $x^n$ and $y^n$ but not on $θ$. A statistic $T(x^n)$ is sufficient if $T(x^n)↔T(y^n)$ implies that $x^n↔y^n$.

  • it is important to distinguish random objects like $X^N$ from their realisation, like $x^n$. The relation $x^n↔y^n$ holds between a pair of realisations,
  • the density of $X^n$ should be written $f_n(x^n;\theta)$ since the dimension of the entry $x^n\in\mathfrak X^n$ depends on $n$,
  • the proportionality equation $f(x^n;θ)=cf(y^n;θ)$ should be written more rigorously as $$f(x^n;θ)=c(x^n,y^n)f(y^n;θ)\quad\forall\,\theta\in\Theta$$or, alternatively, as the fact that the mapping$$\theta\longmapsto f(x^n;θ)\big/ f(y^n;θ)$$is constant,
  • the statistic $T(X^n)$ should be indexed by $n$ as well, since it maps $\mathfrak X^n$ to another space $\mathfrak T_n$ that may depend on $n$. (In the Uniform example, it does not since $\mathfrak T_n=\mathbb R_+$ for all $n$'s.)
  • $T(x^n)↔T(y^n)$ means that$$f_{T_n}(t;\theta)=c(t,t^\prime)f_{T_n}(t^\prime;\theta)\quad\forall\,\theta\in\Theta$$ when $t=T_n(x^n)$, $t^\prime=T_n(y^n)$ and $f_{T_n}(t;\theta)$ is the density of $T_n(X^n)$, which is not the density of the original random object $X^n$ but its so-called push-forward image by $T_n$,
  • the definition thus means that, if $T_n$ is sufficient, then, when the mapping $$\theta\longmapsto f_{T_n}(T_n(x^n);\theta)\big/ f_{T_n}(T_n(y^n);\theta)$$is constant, the mapping $$\theta\longmapsto f(x^n;\theta)\big/ f(y^n;\theta)$$is necessarily constant

When considering the illustrative Uniform case, if $T_n(x^n)=\max_{1\le i\le n}x_i$, the density of $T_n(X^n)$ is $$f_{T_n}(t;\theta)=\frac{nt^{n-1}}{\theta^n}\mathbb I_{(0,\theta)}(t)$$ If $(t,t^\prime)$ is such that $$f_{T_n}(t;\theta)=c(t,t^\prime)f_{T_n}(t^\prime;\theta)\quad\forall\,\theta\in\Theta$$it implies that $$\mathbb I_{(0,\theta)}(t)=\mathbb I_{(0,\theta)}(t^\prime)\quad\forall\,\theta\in\Theta$$hence that $t=t^\prime$. And if $T_n(x^n)=T_n(y^n)$, the mapping $$\theta\longmapsto f(x^n;θ)=\theta^{-n}\prod_{i=1}^n \mathbb I_{(0,\theta)}(x_i) =\theta^{-n}\mathbb I_{\theta\ge T_n(x^n)}=\theta^{-n}\mathbb I_{\theta\ge T_n(y^n)}=f(y^n;\theta)$$is the same for $x^n$ and $y^n$, therefore $x^n↔y^n$ and $T_n(X^n)$ is sufficient for all $n\ge 1$'s.

$\endgroup$
5
  • 2
    $\begingroup$ Thank you for supplying much needed clarity on the matter. $\endgroup$ – microhaus Mar 28 at 13:02
  • 2
    $\begingroup$ This really should be the answer that is accepted. Might it be a good idea to delete my extended comment? I am concerned that those self-studying who browse the Q & A run the risk of potentially being misled by the fact that I have implicitly reinforced erroneous reasoning by writing it out, and by the fact that it has a huge green tick beside it. However, it does leave a 'paper trail'. $\endgroup$ – microhaus Mar 28 at 13:08
  • 2
    $\begingroup$ @microhaus: thank you. I do not think your answer should be removed for you place several warnings along it that this is a comment. Besides, you should feel free to update this answer by correcting the erroneous parts. $\endgroup$ – Xi'an Mar 28 at 14:13
  • $\begingroup$ @Xi'an thank you for taking the time to provide a detailed answer. I tried following the hint you gave previously but I got stuck when I found $\mathbb{I}_{(0,\theta)}(t) = \mathbb{I}_{(0,\theta)}(t')$. I wasn't sure where to go from there, but as you said in your answer it implies $t=t'$. This definition is in the appendix of Ch.9 in my textbook, and it almost seems as an afterthought because, as you mentioned, many details of how to interpret the definition are left unsaid. $\endgroup$ – nwsteg Mar 30 at 18:46
  • 1
    $\begingroup$ If the functions$$\theta\mapsto\mathbb I_{(0,1)}(t)$$and$$\theta\mapsto\mathbb I_{(0,1)}(t')$$are equal, they give identical images for all values of $\theta$. This means they are equal to one iff $\theta>t$ and iff $\theta> t'$. The limits $t$ and $t'$ must thus be the same. $\endgroup$ – Xi'an Mar 30 at 19:59
1
$\begingroup$

Extended comment.

I am somewhat confused by what it is you are seeking an answer for, and also by your workings. On the workings:

Assuming that by $f(x^n; \theta)$ you refer to the likelihood function $L(\theta)$, I am unable to understand why you have specified your likelihood function to be

$$L(\theta) = \theta^{-n} \mathbf{1}_{[0, \infty]} (\min x^n) \mathbf{1}_{(-\infty, \theta]}(\max x^n)$$

In particular, at first glance I am not entirely sure if you are specifying the condition under which $L(\theta)$ is nonzero correctly. Further, if it is correct, then it would appear to contain some redundancy in the conditions. That is, I am currently unable to see good reason as to why you would not specify the following instead:

$$L(\theta) = \theta^{-n} \mathbf{1}_{(-\infty, \theta]}(\max x^n)$$

Assuming that you are trying to use the definition to show that $T(x^n) = \max \{x_1, ... x_n\}$ is sufficient, aren't you trying to show that assuming the truth of the statement $T(x^n) \leftrightarrow T(y^n)$ implies $x^n \leftrightarrow y^n$? If so, rather than one long chain of equalities, might it not perhaps be more appropriate to present the structure as:

Assume that [...], that is,

$$\begin{align} \theta^{-n} \mathbf{1}_{(-\infty, \theta]}(\max T(x^n)) = c\theta^{-n} \mathbf{1}_{(-\infty, \theta]}(\max T(y^n)) \end{align}$$

Because [...], the above implies that

$$\theta^{-n} \mathbf{1}_{(-\infty, \theta]}(\max x^n) = c\theta^{-n} \mathbf{1}_{(-\infty, \theta]}(\max y^n)$$

Conclude.


Now perhaps this might just be a distinction in study, or indeed self-study practices, but all I make of the definition is that it's merely a formal way of saying that for a statistic to be sufficient, it must be the case that retention of this statistic alone allows you to compute the likelihood function without further recourse to rest of the data. And that solely with respect to this purpose, you can effectively bin the rest of the data and still compute the likelihood function. And that the inclusion of $c$ in the definition is just a formal way of expressing the fact that the likelihood function is really an equivalence class of functions.

As an entirely separate self-study point, but perhaps relevant, when LW goes over sufficiency in the publicly available CMU introductory theoretical statistics course here and here, he didn't devote any time on the definition you have quoted from the book, nor any space for it in the lecture notes either; rather goes down the expository route of partitions and conditional distributions.

$\endgroup$
3
  • 2
    $\begingroup$ To show that $T(x^n) \leftrightarrow T(y^n)$ using AoS definition, one need first derive the density of $T(X^n)$. $\endgroup$ – Xi'an Mar 27 at 8:16
  • $\begingroup$ @Xi'an thank you for pointing this out. I was following along with the OP's logic and I was clearly not sufficiently critical nor attentive. $\endgroup$ – microhaus Mar 27 at 23:57
  • $\begingroup$ @StrugglingStudent42 I do not have the powers of which you speak $\endgroup$ – microhaus Mar 27 at 23:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.