Skip to main content

Questions tagged [sufficient-statistics]

A sufficient statistic is a lower dimensional function of the data which contains all relevant information about a certain parameter in itself.

Filter by
Sorted by
Tagged with
22 votes
3 answers
7k views

Jointly Complete Sufficient Statistics for Uniform$(a, b)$ Distributions

Let $\mathbf{X}= (x_1, x_2, \dots x_n)$ be a random sample from the uniform distribution on $(a,b)$, where $a < b$. Let $Y_1$ and $Y_n$ be the largest and smallest order statistics. Show that ...
emlu's user avatar
  • 221
6 votes
1 answer
8k views

Sufficient statistics for Uniform $(-\theta,\theta)$

So, I know that $\max(-X_{(1)},X_{(n)})$ is a sufficient statistic for the parameter $\theta$. But can I also say that $(X_{(1)},X_{(n)})$ are jointly sufficient for the parameter $\theta$ ? In other ...
Tricolor's user avatar
  • 307
10 votes
1 answer
7k views

What does it mean that a statistic $T(X)$ is sufficient for a parameter?

I am having a hard time understanding what a sufficient statistic actually helps us do. It says that Given $X_1, X_2, ..., X_n$ from some distribution, a statistic $T(X)$ is sufficient for a parameter ...
user123276's user avatar
  • 2,107
8 votes
2 answers
643 views

Is there a difference between Bayesian and Classical sufficiency?

The title pretty much says it all. I wonder whether there is any difference in the way Bayesians understand sufficiency vs. the way orthodox statistics understands sufficiency, or are they equivalent? ...
Sebastian's user avatar
  • 3,104
33 votes
2 answers
2k views

When if ever is a median statistic a sufficient statistic?

I came across a casual remark on The Chemical Statistician that a sample median could often be a choice for a sufficient statistic but, besides the obvious case of one or two observations where it ...
Xi'an's user avatar
  • 106k
12 votes
3 answers
4k views

Solution to German Tank Problem

Is there a formal mathematical proof that the solution to the German Tank Problem is a function of only the parameters k (number of observed samples) and m (maximum value among observed samples)? In ...
Bogdan Alexandru's user avatar
23 votes
7 answers
3k views

Why does a sufficient statistic contain all the information needed to compute any estimate of the parameter?

I've just started studying statistics and I can't get an intuitive understanding of sufficiency. To be more precise I can't understand how to show that the following two paragraphs are equivalent: ...
gcoll's user avatar
  • 381
13 votes
2 answers
1k views

How does Bayesian Sufficiency relate to Frequentist Sufficiency?

The simplest definition of a sufficient statistics in the frequentist perspective is given here in Wikipedia. However, I recently came across in a Bayesian book, with the definition $P(\theta|x,t)=P(\...
An old man in the sea.'s user avatar
6 votes
1 answer
2k views

Basic intuition about minimal sufficient statistic

As stated by Wikipedia: A sufficient statistic is minimal sufficient if it can be represented as a function of any other sufficient statistic. In other words, $S(X)$ is minimal sufficient if and ...
Lex's user avatar
  • 269
5 votes
1 answer
9k views

When using the likelihood function, where does the indicator function come from?

For finding sufficient estimators and MLE's, there are certain distributions that require the indicator function such as the uniform distribution. Where does it come from in simple language? and for ...
Jaider's user avatar
  • 161
4 votes
1 answer
773 views

Can this statistic be shown not to be sufficient for $\theta$?

This problem comes from Casella and Berger, who do not rigorously demonstrate (in their solution key) that the statistic is not sufficient. Let $X_1,\dots,X_n$ be a random sample from a population ...
FakeAnalyst56's user avatar
3 votes
1 answer
10k views

Sufficient statistic for bivariate or multivariate normal

Here on page 7, example 2.7. The claim is that sufficient statistics for $d$ dimensional multivariate normal $\mathbf{x}_i \sim N(\vec{\mu}, \Sigma)$ is $$\left(n^{-1}\sum_{i=1}^n \mathbf{x}_i, \hat{\...
them's user avatar
  • 702
31 votes
3 answers
4k views

Sufficient statistics for layman

Can someone please explain sufficient statistics in very basic terms? I come from an engineering background, and I have gone through a lot of stuff but failed to find an intuitive explanation.
user1343318's user avatar
  • 1,351
12 votes
1 answer
510 views

Sufficiency or Insufficiency

Consider a random sample $\{X_1,X_2,X_3\}$ where $X_i$ are i.i.d. $Bernoulli(p)$ random variables where $p\in(0,1)$. Check if $T(X)=X_1+2X_2+X_3$ is a sufficient statistic for $p$. Firstly, how ...
Landon Carter's user avatar
10 votes
1 answer
482 views

Are unbiased efficient estimators stochastically dominant over other (median) unbiased estimators?

General description Does an efficient estimator (which has sample variance equal to the Cramér–Rao bound) maximize the probability for being close to the true parameter $\theta$? Say we compare the ...
Sextus Empiricus's user avatar
10 votes
2 answers
2k views

Find the unique MVUE

This question is from Robert Hogg's Introduction to Mathematical Statistics 6th Version problem 7.4.9 at page 388. Let $X_1,...,X_n$ be iid with pdf $f(x;\theta)=1/3\theta,-\theta<x<2\theta,$ ...
Deep North's user avatar
  • 4,746
8 votes
1 answer
2k views

Minimum dimension of sufficient statistics

Suppose that we have a parameter of $k$-dimensions. Say, for example, for $N(u,\theta)$ both unknown then the parameter is of two dimensions, and $n$ i.i.d. observations. Is it possible to find a ...
nobody's user avatar
  • 611
8 votes
2 answers
2k views

Puzzled by the definition of sufficient statistics in Mood, Graybill, and Boes

I am learning about sufficient statistic from Mood, Graybill, and Boes's Introduction to the Theory of Statistics. I am slightly confused by the book's definition of a sufficient statistic for ...
Noppawee Apichonpongpan's user avatar
8 votes
1 answer
5k views

Are complete statistics always sufficient?

I know that a complete sufficient statistic $T$ is such that 1) $T$ is sufficient for $\theta$, unknown parameter and 2) $T$ is complete. So, is it always the case? If the answer is not, what ...
PhDing's user avatar
  • 3,099
5 votes
1 answer
8k views

Proof that n-order statistics are sufficient for a sample of size n

This is problem 1.5.8 in Mathematical Statistics by Bickel and Doksum. It seems straightforward, but I am not sure if my proof is lacking in some way. It doesn't seem quite correct. Question Let $...
fogofwar's user avatar
4 votes
1 answer
1k views

Conditional distribution of $(X_1,\cdots,X_n)\mid X_{(n)}$ where $X_i$'s are i.i.d $\mathcal U(0,\theta)$ variables

Let $(X_1,X_2,\cdots,X_n)$ be a random sample drawn from $\mathcal U(0,\theta)$ distribution. It is a common exercise to prove that the maximum order statistic $X_{(n)}$ is sufficient for $\theta$ as ...
StubbornAtom's user avatar
  • 11.4k
3 votes
1 answer
442 views

Prove that the sum is sufficient using using the definition of sufficiency

If $X_1,\ldots,X_n$ is an IID random sample, with $X_i\sim\,\text{Ber}(\theta)$, prove that $Y = \sum_i X_i$ is sufficient using the definition of sufficiency (not the factorization criterion). Now ...
laurab's user avatar
  • 145
16 votes
1 answer
2k views

Sufficient statistic, specifics/intuition problems

I'm teaching myself some statistics for fun and I have some confusion regarding sufficient statistics. I'll write out my confusions in list format: If a distribution has $n$ parameters then will it ...
Kimchi's user avatar
  • 161
11 votes
1 answer
2k views

Proof of Pitman–Koopman–Darmois theorem

Where can I find a proof of Pitman–Koopman–Darmois theorem? I have googled for quite some time. Strangely, many notes mention this theorem yet none of them present the proof.
3x89g2's user avatar
  • 1,716
10 votes
1 answer
3k views

Complete sufficient statistic

I've recently started studying statistical inference. I've been working through various problems and this one has me completely stumped. Let $X_1,\dots,X_n$ be a random sample from a discrete ...
Tony's user avatar
  • 101
9 votes
1 answer
1k views

The minimal sufficient statistic of $f(x) = e^{-(x-\theta)}e^{-e^{-(x-\theta)}}$

The Casella Berger (2002) solutions manual says that the minimal sufficient statistic for $$f(x) = e^{-(x-\theta)}e^{-e^{-(x-\theta)}}, \qquad x\in \mathbb{R}$$ are the order statistics $(X_{(1)},\...
Xiaomi's user avatar
  • 2,554
9 votes
2 answers
2k views

Likelihood Function is Minimal Sufficient

What does it mean to say that "Likelihood Function is Minimal Sufficient"? Is this a general statement, or does it apply to only exponential family of distributions? I think I understand the concept ...
Cagdas Ozgenc's user avatar
9 votes
3 answers
849 views

Sufficient Statistics - Relating the Intuition with the Mathematical Definition

I believe the heuristic definition of a Sufficient Statistic makes sense to me - when you take a sample in order to make an inference about the parameter related to the probability distribution, and ...
user523384's user avatar
8 votes
3 answers
7k views

Is a minimal sufficient statistic also a complete statistic

I know that if a statistic is both sufficient and complete then it must also be minimal sufficient. But on the other hand, could I say a minimal sufficient statistic must also be a complete statistic?
zqzwxec11's user avatar
  • 101
8 votes
1 answer
7k views

What is exponential family criterion to test the sufficiency and completeness of an estimator?

I am struggling to understand the following result from Casella and Berger about sufficiency and completeness for exponential families: Let $X_{1},X_{2},...,X_{n}$ be iid observations from an ...
userNoOne's user avatar
  • 1,048
7 votes
1 answer
2k views

Whether the minimal sufficient statistic is complete for a translated exponential distribution

Let $X_1, X_2..., X_n$ follows iid negative exponential distribution with pdf $$f(x) = \frac{1}{\theta^2} \: e^{-\frac{(x-\theta)}{\theta^2}} \: \: I_{(x>\theta)} $$ I have to show whether the ...
n1234's user avatar
  • 73
7 votes
1 answer
1k views

Relationship between completeness and sufficiency

hopefully this isn't a duplicate of another question (at least I didn't find one). Here is a question I have about completeness and sufficiency: Problem: Suppose $T(x)$ is complete sufficient for $\...
asdf's user avatar
  • 384
6 votes
2 answers
2k views

Finding complete sufficient statistic

Let $X_1, \dots, X_n$ be iid. $\text{Uniform}[-\theta,\theta]$. I need to find the complete sufficient statistic for $\theta$ or prove there does not exist such. I know that $T = (X_{(1)}, X_{(n)} )$ ...
Sam88's user avatar
  • 348
6 votes
1 answer
563 views

Trying to make sense of claims regarding Rao-Blackwell and Lehmann-Scheffé for sufficient/complete statistics

I am currently trying to learn the two related concepts of the Rao-Blackwell theorem and the Lehmann-Scheffé theorem. Assume we have the random sample $X_1, \dots, X_n$ with mean $\mu$ and variance $\...
The Pointer's user avatar
  • 2,022
6 votes
1 answer
4k views

Are the order statistics minimal sufficient for a location-scale family?

Suppose we have a location-scale family with pdf $$\frac{1}{\sigma}f(\frac{x - \mu}{\sigma}).$$ Let $X_1$, $X_2$, $X_3$,..., $X_n$ be a random sample from the location-scale family. Is the statistics $...
Noppawee Apichonpongpan's user avatar
5 votes
2 answers
979 views

Intuitive and Formulaic Justification for the Rao-Blackwell Theorem

I've tried looking online and I can't seem to grasp the Rao-Blackwell theorem. Could someone please give an intuitive explanation backed up by formulae.
AER's user avatar
  • 205
4 votes
2 answers
914 views

If the dimension of a sufficient statistic $T(X)$ equals the dimension of parameter space, $T(X)$ is minimal sufficient?

I came cross an interesting comment saying If the dimension of a sufficient statistic $T(X)$ is the same as that of the parameter space, then $T(X)$ is minimal sufficient. Is this is true? I ...
Tan's user avatar
  • 1,489
4 votes
1 answer
251 views

What are the "Dangers" of using "Non-Sufficient" Statistics?

I was reading one of the answers listed on this previous Stackoverflow question about the importance of sufficient statistics (Generalized Linear Models - What's special about the exponential ...
stats_noob's user avatar
4 votes
2 answers
1k views

Sufficient statistic for Gaussian $AR(1)$

Question Does the Gaussian $AR(1)$ model, with a fixed sample size $T$, have nontrivial sufficient statistics? The model is given by $$ y_t = \rho y_{t-1}, \, t = 1, \cdots, T, \; \epsilon_i \...
Michael's user avatar
  • 3,348
3 votes
1 answer
2k views

A sufficient statistic for Laplace distribution

Suppose we have p dimensional vector of $X =[X_1 \dots X_n]$ where X is Laplace distributed. What will be a sufficient statistics for estimating covariance of $X$? Would it be the sample covariance,...
undefined's user avatar
  • 463
3 votes
1 answer
2k views

Sufficient statistic for function of parameter

Suppose we are estimating $\tau(\theta)=\theta e^{-\theta}$ from $X_1,...,X_n \sim \mathrm{G}(\theta,r)$ (G is the gamma distribution) then it is easily shown that $T=\sum_{i=1}^n \ln(X_i)$ is ...
Raxel's user avatar
  • 347
2 votes
1 answer
308 views

How do we conclude that a statistic is sufficient but not minimal sufficient?

I want to show that the statistic $\left(\sum_{i = 1}^n Y_i, \sum_{i = 1}^n Y_i^2 \right)$ is sufficient for $\mu$ but not minimal sufficient where $(Y_1, \dots, Y_n)$ is a random sample from $N(\mu, \...
The Pointer's user avatar
  • 2,022
2 votes
2 answers
509 views

Can the Fisher factorization theorem be understood as a product of densities?

Let $T$ be some random variable on a probability space $\Omega$. Then we have, for $x\in\Omega$: $$P(x) = P(x|T=T(x))P(T = T(x))$$ This equation is nonsense in an arbitrary probability space but ...
Jack M's user avatar
  • 439
2 votes
1 answer
1k views

Intuition for why the sum of Bernoulli trials is a sufficient statistic via cancellations on numerator and denominator?

Suppose that $X_1, \ldots, X_n$ are iid Bernoulli trials with parameter $p$, and that $Y = \sum_{i=1}^nX_i$. Then, we have that $Y$ is a sufficient statistic by the following condensed proof: $$ P(...
user321627's user avatar
  • 4,448
2 votes
1 answer
3k views

How to find sufficient complete statistic for the density $f(x\mid\theta)=e^{-(x-\theta)}\exp(-e^{-(x-\theta)})$?

For the pdf $$f(x\mid\theta)=e^{-(x-\theta)}\exp(-e^{-(x-\theta)})\,,\qquad-\infty<x<\infty,-\infty<\theta<\infty$$, let $X_1,\cdots,X_n$ be i.i.d observations. How to find the sufficient ...
Bonnie Gu's user avatar
2 votes
2 answers
240 views

Sufficiency of $|X|$ when $X\sim N(0,\sigma^2)$ without using Factorization theorem

Question: Given, $X\sim N(0,\sigma^2)$. By means of conditional approach show that $|X|$ is a sufficient estimator for $\sigma^2$. My Attempt: This problem is very easy if we use Fisher–Neyman ...
RATNODEEP BAIN's user avatar
2 votes
2 answers
2k views

Probability conditional on a parameter?

This is a definition of the sufficient statistic from Wikipedia. A statistic $t = T(X)$ is sufficient for underlying parameter $θ$ precisely if the conditional probability distribution of the data $...
rimusolem's user avatar
1 vote
1 answer
385 views

Unbiased Estimator based on Sufficient Statistic

suppose $X_1, ... , X_n$ are iid with pdf $f(x|\beta) = e^{-(x-\beta))}I_{(\beta, \infty)}(x)$ and the pdf of ( the smallest order statistic) $X_{(1)}$ is given by $f_{X_1}(x)$ = n $ *$ $e^{n(\...
Pedros's user avatar
  • 213
0 votes
1 answer
133 views

Doubt Regarding the Sufficient Statistic Problem

Let $X_1, X_2, ..., X_n$ be random variables from the following densities: $$ f(x_i|\theta) = \frac{1}{2i\theta} \text{ for } -i(\theta-1) < x_i < i(\theta+1) $$ and otherwise the density is ...
userNoOne's user avatar
  • 1,048
30 votes
2 answers
3k views

Why do we not care about completeness, sufficiency of an estimator as much anymore?

When we begin to learn Statistics, we learn about seemingly important class of estimators that satisfy the properties sufficiency and completeness. However, when I read recent articles in Statistics I ...
pineapple's user avatar
  • 413