Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Can someone please explain sufficient statistics in very basic terms? I come from an engineering background, and I have gone through a lot of stuff but failed to find an intuitive explanation.

share|improve this question

2 Answers 2

up vote 18 down vote accepted

A sufficient statistic summarizes all the information contained in a sample so that you would make the same parameter estimate whether we gave you the sample or just the statistic itself. It's reduction of the data without information loss.

Here's one example. Suppose $X$ has a symmetric distribution about zero. Instead of giving you a sample, I hand you a sample of absolute values instead (that's the statistic). You don't get to see the sign. But you know that the distribution is symmetric, so for a given value $x$, $-x$ and $x$ are equally likely (the conditional probability is $0.5$). So you can flip a fair coin. If it comes up heads, make that $x$ negative. If tails, make it positive. This gives you a sample from $X'$, which has the same distribution as the original data $X$. You basically were able to reconstruct the data from the statistic. That's what makes it sufficient.

share|improve this answer
    
Thanks, Dimitriy for that. –  user1343318 Nov 21 '12 at 21:37

In Bayesian terms, you have some observable property $X$ and a parameter $\Theta$. The joint distribution for $X,\Theta$ is specified, but factored as the conditional distribution of $X\mid \Theta$ and the prior distribution of $\Theta$. A statistic $T$ is sufficient for this model if and only if the posterior distribution of $\Theta\mid X$ is the same as that of $\Theta\mid T(X)$, for every prior distribution of $\Theta$. In words, your updated opinion about $\Theta$ after knowing the value of $X$ is the same as your updated opinion about $\Theta$ after knowing the value of $T(X)$, whatever prior opinion you have about $\Theta$. Keep in mind that sufficiency is a model dependent concept.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.