4
$\begingroup$

So, I know that $\max(-X_{(1)},X_{(n)})$ is a sufficient statistic for the parameter $\theta$. But can I also say that $(X_{(1)},X_{(n)})$ are jointly sufficient for the parameter $\theta$ ?

In other words, can a single parameter have jointly sufficient statistics?

$\endgroup$
  • 6
    $\begingroup$ Yes you can have jointly sufficient statistics. In fact the order statistics $T=(X_{(1)}, X_{(2)}, \cdots X_{(n)})$ are always sufficient (somewhat trivially). What you lack however, is minimal sufficiency (and in this case completeness), a rather important property for many applications. $\endgroup$ – knrumsey Jul 6 '18 at 21:03
  • 1
    $\begingroup$ Your question is not very clear, but note that if $(T_1, T_2)$ is sufficient, then so will be $g(T_1, T_2)$ when $g$ is a one-to-one function. $\endgroup$ – kjetil b halvorsen Jul 6 '18 at 23:42
  • 2
    $\begingroup$ @knrumsey comments that $(X_{(1)}, X_{(n)})$ will not be minimally sufficient (unless $n=1$). That is because in this peculiar model, we have another sufficient statistic, $\max\left( |X_{(1)}|, |X_{(n)}| \right)$. $\endgroup$ – kjetil b halvorsen Jul 6 '18 at 23:52
  • 2
    $\begingroup$ In fact, in some families like $N(\theta,\theta^2)$, the dimension of the sufficient statistic is two while the parameter $\theta$ is one-dimensional. But that is not the case here. In the Uniform family, we often get $(X_{(1)},X_{(n)})$ as the two-dimensional sufficient statistic, like in $U(a,b)$ where both $a,b$ are unknown and even in $U(\theta,\theta+1)$ where $\theta$ is the single parameter. $\endgroup$ – StubbornAtom Jul 7 '18 at 8:05
  • 1
    $\begingroup$ No, the argument is more involved! But note that in $U(-\theta,\theta)$ there is a symmetry not present in $U(a,b)$ or $U(\theta,\theta+1)$. $\endgroup$ – kjetil b halvorsen Jul 7 '18 at 8:12
6
$\begingroup$

Suppose we have a random sample $(X_1,X_2,\cdots,X_n)$ drawn from $\mathcal U(-\theta,\theta)$ distribution.

PDF of $X\sim\mathcal U(-\theta,\theta)$ is $$f(x;\theta)=\frac{1}{2\theta}\mathbf1_{-\theta<x<\theta},\quad\theta>0$$

Joint density of $(X_1,X_2,\cdots,X_n)$ is

\begin{align}f_{\theta}(x_1,x_2,\cdots,x_n)&=\prod_{i=1}^nf(x_i;\theta) \\&=\frac{1}{(2\theta)^n}\mathbf1_{-\theta<x_1,\cdots,x_n<\theta} \\&=\frac{1}{(2\theta)^n}\mathbf1_{0<|x_1|,\cdots,|x_n|<\theta} \\&=\frac{1}{(2\theta)^n}\mathbf1_{\max_{1\le i\le n}|x_i|<\theta} \end{align}

To clearly use the factorisation theorem, let us define $\mathbb I(x)=\begin{cases}1&,\text{ if }x>0\\0&,\text{ otherwise }\end{cases}$

Then we have

\begin{align} f_{\theta}(x_1,x_2,\cdots,x_n)&=\frac{1}{(2\theta)^n}\mathbb I\left(\theta-\max_{1\le i\le n}|x_i|\right) \\&=g\left(\theta,\max_{1\le i\le n}|x_i|\right)h(x_1,x_2,\cdots,x_n) \end{align}

where $g\left(\theta,\max_{1\le i\le n}|x_i|\right)=\frac{1}{(2\theta)^n}\mathbb I\left(\theta-\max_{1\le i\le n}|x_i|\right)$ depends on $\theta$ and on $x_1,x_2,\cdots,x_n$ through $\max_{1\le i\le n}|x_i|$, and $h(x_1,x_2,\cdots,x_n)=1$ is independent of $\theta$.

So by factorisation theorem, $\max_{1\le i\le n}|X_i|$ is a sufficient statistic for $\theta$.

In fact, it can be shown to be minimal sufficient for $\theta$.

Since $-\theta<x_i<\theta\implies \theta>\max(-x_{(1)},x_{(n)})$, you are correct that $\max(-X_{(1)},X_{(n)})$ is a sufficient statistic for $\theta$ by a similar logic. In fact, if I am not wrong, it can be shown that $$\max(-X_{(1)},X_{(n)})=\max_{1\le i\le n}|X_i|$$

It is perfectly valid for a single unknown parameter to have jointly sufficient statistics since the definition of sufficiency tells us that a set of statistics $T_1(X_1,\cdots,X_n),\cdots,T_k(X_1,\cdots,X_n)$ is jointly sufficient for $\theta$ (which may be a vector) if and only if the conditional distribution of $X_1,\cdots,X_n$ given $T_1,\cdots,T_k$ does not depend on $\theta$.

As for example, we always have two trivial choices of jointly sufficient statistics. One is the sample $(X_1,X_2,\cdots,X_n)$ itself and the other is the set of order statistics $(X_{(1)},X_{(2)},\cdots,X_{(n)})$ as mentioned by knrumsey in the comments.

Again, $(X_{(1)},X_{(n)})$ is also sufficient for $\theta$ as it has the two components $X_{(1)}$ and $X_{(n)}$ of the minimal sufficient statistic, but it is not itself minimal sufficient as mentioned in the comments.


To clarify further queries of OP in the comments:

If a statistic $T$ is sufficient for a one-dimensional parameter $\theta$, then $\mathbf T=(T,T_0)$ will also be a sufficient statistic for $\theta$ for any other statistic $T_0$. This is because $\mathbf T$ can be looked as a one-to-one function of $T$. In particular, if $T$ is minimal sufficient, then $\mathbf T$ will be sufficient in a sense that the second component $T_0$ in $\mathbf T$ will be of no use in further data condensation and data reduction without loosing any information about $\theta$. So $T$ will be preferred to $(T,T_0)$ as a sufficient statistic.

For example, consider the $\mathcal N(\theta,1)$ population. Here the sample mean $\bar X$ is minimal sufficient for $\theta$. Now if we take $\mathbf T=(\bar X,S^2)$ where $S^2$ is the sample variance, then $\mathbf T$ will remain sufficient but no longer minimal sufficient.

Thanks to @knrumsey for pointing out my error.

$\endgroup$
  • $\begingroup$ So, Imagine that I know that $\sum X_{i}$ is sufficient for a certain paramether $\theta$. Can I also say that $(\sum X_{i}, X_{(a)}) $ (where $X_{(a)}$ is any order statistics) is jointly sufficient for the same paramether? In other words,can I manipulate the factorisation theorem in such way that any order statistics or even the sample itself appears within the part of the function that depends on both the sample and $\theta$ (meaning they will be sufficient for $\theta$)?. I'm not sure if the second part is very clear, since english is not my primary language. Thanks for the answers! $\endgroup$ – Tricolor Jul 7 '18 at 22:06
  • $\begingroup$ @Tricolor I don't think that's true in general. Say we consider the Bernoulli$(\theta)$ distribution. Then $T(X_1,\cdots,X_n)=\sum_{i=1}^n X_i$ is a minimal sufficient statistic for $\theta$. In other words, $T$ is itself a function of every other set of sufficient statistics. As in, $(X_{(1)},\cdots,X_{(n)})$ is jointly sufficient for $\theta$, and $T=\sum_{i=1}^nX_i=\sum_{i=1}^n X_{(i)}$ is a function of $X_{(1)},\cdots,X_{(n)}$. $\endgroup$ – StubbornAtom Jul 8 '18 at 8:10
  • $\begingroup$ @Tricolor When you factor the joint density, do you get a $g$ as in my answer where $g$ depends on $x_1,\cdots,x_n$ through $(T,X_{(i)})$? If you don't, then you can be certain that your claim does not hold. In the above answer, the particular pair $(X_{(1)},X_{(n)})$ is a sufficient statistic because the minimal sufficient statistic $\max(-X_{(1)},X_{(n)})$ contains the components $X_{(1)}$ and $X_{(n)}$. That doesn't mean we can incorporate any particular order statistic along with a sufficient statistic to get a jointly sufficient statistic. $\endgroup$ – StubbornAtom Jul 8 '18 at 8:10
  • 2
    $\begingroup$ If $X_{(n)}$ is sufficient for $b$, then so is $(T_0, X_{(n)})$ for any statistic $T_0$. As pointed out by @kjetilbhalvorsen, this is because $g(x) = (a,x)$ is a one to one function from $\mathbb R$ to $\mathbb R^2$. Again, what you lose is not sufficiency, but minimal sufficiency. $\endgroup$ – knrumsey Jul 9 '18 at 4:30
  • 1
    $\begingroup$ @Tricolor Please see the above comments and my edit. $\endgroup$ – StubbornAtom Jul 9 '18 at 10:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.