The definition of a sufficient statistic is: Let $X_1,...,X_n$ be a random sample from a distribution indexed by a parameter $\theta$. Let $T$ be a statistic. Suppose that, for every $\theta$ and every possible value $t$ of $T$, the conditional joint distribution of $X_1,...,X_n$ given that $T=t$ depends only on $t$ but not on $\theta$. Then, $T$ is a sufficient statistic for parameter $\theta$.

I feel like I know several pieces of the puzzle (like the factorization theorem) to understanding sufficient statistics but do not have the overall theory down.

My main questions are:

1) Why do they say that $T$ is a sufficient statistic for parameter $\theta$? If $\theta$ were the population mean of a normal distribution, say $\mu$, does it mean that anytime we want to find the probability of, say, $X_1,...,X_n$ occurring in a certain way, that we don't need the value of the mean of the population?

2) In real-life, why do we want to use a sufficient statistic? Is seems that just calculating the statistic shouldn't be that much work (such as the sum of X's) so why do we need it?



1 Answer 1

  1. No. What they say is if $X_1^\prime,\dots,X_n^\prime$ is another random sample from the same population as the original data $X_1,\dots,X_n$, it contains an equal amount of probabilistic information about $\theta$. Therefore, we can "recover the data" if we retain $T$ and discard $X_1,\dots,X_n$. That’s why $T$ is "sufficient".

  2. Data reduction. If $T$ is sufficient, the "extra information" carried by $X$ is worthless as long as $θ$ is concerned. It is then only natural to consider inference procedures which do not use this extra irrelevant information. This leads to the Sufficiency Principle: Any inference procedure should depend on the data only through sufficient statistics.

See here for more detail on principles involved in data reduction.

  • $\begingroup$ The link www4.stat.ncsu.edu/~hzhang/st522/08chap6.pdf is forbidden. $\endgroup$
    – mon
    Commented Dec 17, 2022 at 7:59
  • $\begingroup$ "... it contains an equal amount of probabilistic information about 𝜃" - I think you are conflating F-Sufficiency and B-Sufficiency. The question defines F-Sufficient statistic that makes no comment on the "probabilistic information about 𝜃", it only claims that probabilistic information about the sample, given the statistic, is independent of 𝜃. $\endgroup$
    – Shreyans
    Commented May 26 at 11:50
  • $\begingroup$ "This leads to the Sufficiency Principle" - You start with Sufficiency principle when you claim "If 𝑇 is sufficient, the 'extra information' carried by 𝑋 is worthless as long as θ is concerned", so I don't see how it "leads" to Sufficiency Principle. ie I think the argument is circular. $\endgroup$
    – Shreyans
    Commented May 26 at 11:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.