Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 3 down vote favorite

I have a question:

Does a sufficient statistic have to be one to one? For example, can $T(x) = x^2$ or $T(x) = |x|$ be sufficient statistics? I know that one to one functions of sufficient statistics are sufficient.

share|improve this question
2  
Hint: Can you think of an example where this is the case, say, in the context of a family of normal distributions? How would you prove it? – cardinal Apr 9 '12 at 14:02

2 Answers

up vote 2 down vote

No, it doesn't have to be one-to-one. Consider a distribution family with pmf

$$f(x \mid \theta) = \begin{cases} 0 & \text{with probability 0.5 if } \theta=0 \\ 1 & \text{with probability 0.5 if } \theta=0 \\ 2 & \text{with probability 0.5 if } \theta=1 \\ 3 & \text{with probability 0.5 if } \theta=1 \\ \end{cases}$$

Then, the sufficient statistic for $x$ is $\left\lfloor\frac x2\right\rfloor$.

The way I think of sufficient statistics is that they tell you what you learned from $x$ to identify $\theta$.

When you don't use all of the information in a single observation $x$, that is, if there exists two outcomes $\omega_1, \omega_2$ such that $T(\omega_1) = T(\omega_2)$, then there is no way to alter the parameters $\theta$ of the distribution to change the relative probabilities $f(\omega_1 \mid \theta) / f(\omega_2 \mid \theta)$ of the two outcomes.

share|improve this answer
1  
(+1) Good example. It might be slightly unconventional in the sense that from the sufficient statistic in this example, one can recover perfect knowledge of the parameter of interest. It might be helpful to have a second one that uses precisely the proposed statistic that the OP gives, as well. – cardinal Apr 9 '12 at 14:06
@cardinal: Good point. Do you think that would be a good exercise for the OP? – Neil G Apr 9 '12 at 14:07
1  
Yes, I think it would make a good exercise. If it's ok, allow me to quibble just a little bit with your last sentence. Usually the point of sufficient is precisely because we don't want to use all of the information in $x$. In other words, in a multiple-observation scenario, the vector $(X_1,\ldots,X_n)$ is always sufficient, but we'd rather use some function $T : \mathbb R^n \to \mathbb R^k$ of the vector that maps it down to a smaller space (i.e., $k < n$) without losing information about the parameter. :) – cardinal Apr 9 '12 at 14:19
@cardinal: You're right. I was trying to make a point about when you have a single observation $x$. – Neil G Apr 9 '12 at 14:25
@cardinal: I think I should probably quibble about your quibble though: Usually, $T$ is defined from the sample space to a vector of reals (e.g., $T: \mathbb R \to \mathbb R^k$), and the mapping down to a smaller sample for exponential families is accomplished by finding $E(T(x))$ for $x \in \{x_i\}_{i=1}^n$. (Taking that expectation is almost never one-to-one.) – Neil G Apr 9 '12 at 22:04
show 5 more comments
up vote 1 down vote

I like Neil G's example. I wondered if we could find some simple conditions that would prohibit a sufficient statistics to be one-to-one. Here is my take on this.

(Following the comment by cardinal, in all that follows, suppose that $n\geq 2$.)

Suppose that we have the usual setup: the random variables $X_1,\dots,X_n$ are conditionaly i.i.d., given $\Theta=\theta$, with some density $f_{X_i\mid\Theta}(x_i\mid\theta)$.

As pointed out by cardinal, the identity map $\mathrm{id}:\mathbb{R}^n\to\mathbb{R}^n$ is always sufficient for the parameter $\Theta,$ and it is one-to-one. It is also easy to see that the order statistics $U:\mathbb{R}^n\to\mathbb{R}^n$ defined by $U(x_1,\dots,x_n)=(x_{(1)},\dots,x_{(n)})$ is also sufficient, but it is not one-to-one. The intuitive idea is that the value of both carry the same information about $\Theta$ as the original sample $(x_1,\dots,x_n).$

We generally want our sufficient statistic $T$ to reduce the dimension of the original sample, and this reduction will, in general, as we can see from many of the usual examples, imply that $T$ is not one-to-one.

Now, remember that $T$ is $\textit{minimal}$ sufficient if it is sufficient, and for $\textit{any}$ other sufficient statistic $S,$ if $S(x)=S(y)$, then $T(x)=T(y)$, which means that $T$ is a function of every other sufficient statistic $S$. This definition captures the idea that, if $T$ is minimal sufficient, then you can't find another sufficient statistic that provides more data reduction than $T$ does.

Proposition. If $T$ is minimal sufficient, then $T$ is not one-to-one.

Proof. Suppose that $T$ is minimal sufficient and one-to-one. Let $U$ be the order statistics as defined above. Take a sample point $x=(x_1,\dots,x_n)$, a permutation $\pi:\mathbb{R}^n\to\mathbb{R}^n$, and $y=(x_{\pi(1)},\dots,x_{\pi(n)}),$ such that $x\neq y$. It is clear that $U(x)=U(y)$ and, since $U$ is sufficient, we must have, by definition, that $T(x)=T(y)$, which contradicts the fact that $T$ is one-to-one.

(As pointed out by cardinal below, the writting of the proof can be even shorter.)

Has anyone seem this proposition before? Even stated as an exercise? I'm planning to propose it in a future exam ;-)

share|improve this answer
You'll need at least some minor qualifiers. A bit tongue-in-cheek, admittedly, but, counterexample: Take $n = 1$. – cardinal Apr 10 '12 at 0:42
Yeah! Just hacked your regularity condition. – Zen Apr 10 '12 at 0:53
(+1) This is nice. But, note that it depends on the assumption that $\mathbf X = (X_1,\ldots,X_n)$ for $n \geq 2$ is a random sample, so we should probably state that explicitly (Edit: You did already.). In particular, if we have something like, say, certain generalized linear models (e.g., logistic), then it can turn out that $\mathbf X$ itself is minimal sufficient. – cardinal Apr 10 '12 at 1:00
Also, if you use the usual definition of minimal sufficiency, i.e., $T$ is minimal sufficient if $T$ is sufficient and is a function of every other sufficient statistic, then the proof drops out immediately since $U$ is not one-to-one. – cardinal Apr 10 '12 at 1:03
You're right. The shorter proof stresses the key point: since, in this model, $U$ is sufficent and $\textit{not}$ one-to-one, then the minimal sufficient $T$ can't be one-to-one either. – Zen Apr 10 '12 at 1:13
show 4 more comments

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.