Skip to main content

Questions tagged [sufficient-statistics]

A sufficient statistic is a lower dimensional function of the data which contains all relevant information about a certain parameter in itself.

Filter by
Sorted by
Tagged with
33 votes
2 answers
2k views

When if ever is a median statistic a sufficient statistic?

I came across a casual remark on The Chemical Statistician that a sample median could often be a choice for a sufficient statistic but, besides the obvious case of one or two observations where it ...
Xi'an's user avatar
  • 106k
32 votes
3 answers
3k views

How can a statistician who has the data for a non-normal distribution guess better than one who only has the mean?

Let's say we have a game with two players. Both of them know that five samples are drawn from some distribution (not normal). None of them know the parameters of the distribution used to generate the ...
ryu576's user avatar
  • 2,540
31 votes
3 answers
4k views

Sufficient statistics for layman

Can someone please explain sufficient statistics in very basic terms? I come from an engineering background, and I have gone through a lot of stuff but failed to find an intuitive explanation.
user1343318's user avatar
  • 1,351
30 votes
2 answers
3k views

Why do we not care about completeness, sufficiency of an estimator as much anymore?

When we begin to learn Statistics, we learn about seemingly important class of estimators that satisfy the properties sufficiency and completeness. However, when I read recent articles in Statistics I ...
pineapple's user avatar
  • 413
22 votes
3 answers
7k views

Jointly Complete Sufficient Statistics for Uniform$(a, b)$ Distributions

Let $\mathbf{X}= (x_1, x_2, \dots x_n)$ be a random sample from the uniform distribution on $(a,b)$, where $a < b$. Let $Y_1$ and $Y_n$ be the largest and smallest order statistics. Show that ...
emlu's user avatar
  • 221
20 votes
4 answers
3k views
+50

Why a sufficient statistic contains all the information needed to compute any estimate of the parameter?

I've just started studying statistics and I can't get an intuitive understanding of sufficiency. To be more precise I can't understand how to show that the following two paragraphs are equivalent: ...
gcoll's user avatar
  • 351
16 votes
1 answer
2k views

Sufficient statistic, specifics/intuition problems

I'm teaching myself some statistics for fun and I have some confusion regarding sufficient statistics. I'll write out my confusions in list format: If a distribution has $n$ parameters then will it ...
Kimchi's user avatar
  • 161
14 votes
1 answer
1k views

Intuitive understanding of the Halmos-Savage theorem

The Halmos-Savage theorem says that for a dominated statistical model $(\Omega, \mathscr A, \mathscr P)$ a statistic $T: (\Omega, \mathscr A, \mathscr P)\to(\Omega', \mathscr A')$ is sufficient if (...
Sebastian's user avatar
  • 3,094
13 votes
2 answers
1k views

How does Bayesian Sufficiency relate to Frequentist Sufficiency?

The simplest definition of a sufficient statistics in the frequentist perspective is given here in Wikipedia. However, I recently came across in a Bayesian book, with the definition $P(\theta|x,t)=P(\...
An old man in the sea.'s user avatar
12 votes
1 answer
2k views

What is "Likelihood Principle"?

While I was studying "Bayesian Inference", I happen to encounter the term, "Likelihood Principle" but I don't really get the meaning of it. I assume it is connected to "...
xabzakabecd's user avatar
  • 3,505
12 votes
3 answers
4k views

Solution to German Tank Problem

Is there a formal mathematical proof that the solution to the German Tank Problem is a function of only the parameters k (number of observed samples) and m (maximum value among observed samples)? In ...
Bogdan Alexandru's user avatar
12 votes
1 answer
509 views

Sufficiency or Insufficiency

Consider a random sample $\{X_1,X_2,X_3\}$ where $X_i$ are i.i.d. $Bernoulli(p)$ random variables where $p\in(0,1)$. Check if $T(X)=X_1+2X_2+X_3$ is a sufficient statistic for $p$. Firstly, how ...
Landon Carter's user avatar
12 votes
1 answer
1k views

Do the mean and the variance always exist for exponential family distributions?

Assume a scalar random variable $X$ belongs to a vector-parameter exponential family with p.d.f. $$ f_X(x|\boldsymbol \theta) = h(x) \exp\left(\sum_{i=1}^s \eta_i({\boldsymbol \theta}) T_i(x) - A({\...
Wei's user avatar
  • 345
11 votes
1 answer
641 views

Is there a standard measure of the sufficiency of a statistic?

Given a parametrical model $f_\theta$ and a random sample $X = (X_1, \cdots, X_n)$ from this model, a statistic $T(X)$ is sufficient if the distribution of $X$ given $T(X)$ doesn't depend on $\theta$. ...
Pohoua's user avatar
  • 2,618
11 votes
1 answer
2k views

Proof of Pitman–Koopman–Darmois theorem

Where can I find a proof of Pitman–Koopman–Darmois theorem? I have googled for quite some time. Strangely, many notes mention this theorem yet none of them present the proof.
3x89g2's user avatar
  • 1,716
10 votes
1 answer
7k views

What does it mean that a statistic $T(X)$ is sufficient for a parameter?

I am having a hard time understanding what a sufficient statistic actually helps us do. It says that Given $X_1, X_2, ..., X_n$ from some distribution, a statistic $T(X)$ is sufficient for a parameter ...
user123276's user avatar
  • 2,107
10 votes
1 answer
4k views

Exponential Family: Observed vs. Expected Sufficient Statistics

My question arises from reading reading Minka's "Estimating a Dirichlet Distribution", which states the following without proof in the context of deriving a maximum-likelihood estimator for a ...
Benjamin Bray's user avatar
10 votes
1 answer
596 views

Does a sufficient statistic imply the existence of a conjugate prior?

In the comments on this answer, user Scortchi asks: So iff there's a sufficient statistic of constant dimension, there's a conjugate prior? As far as I know this didn't get a complete answer, so I'm ...
N. Virgo's user avatar
  • 425
10 votes
4 answers
349 views

Why is median not a sufficient statistic? [duplicate]

Suppose a random sample of $n$ variables from $N(\mu,1)$, $n$ odd. The sample median is $M=X_{(n+1)/2}$, the order statistic of the middle of the distribution. How to prove that sample median is not a ...
Diorne's user avatar
  • 101
10 votes
1 answer
3k views

Complete sufficient statistic

I've recently started studying statistical inference. I've been working through various problems and this one has me completely stumped. Let $X_1,\dots,X_n$ be a random sample from a discrete ...
Tony's user avatar
  • 101
10 votes
1 answer
480 views

Are unbiased efficient estimators stochastically dominant over other (median) unbiased estimators?

General description Does an efficient estimator (which has sample variance equal to the Cramér–Rao bound) maximize the probability for being close to the true parameter $\theta$? Say we compare the ...
Sextus Empiricus's user avatar
10 votes
1 answer
398 views

What is the intuitive sense behind the purpose and mechanics of Sufficient Statistics?

The definition of a sufficient statistic is: Let $X_1,...,X_n$ be a random sample from a distribution indexed by a parameter $\theta$. Let $T$ be a statistic. Suppose that, for every $\theta$ and ...
user123276's user avatar
  • 2,107
10 votes
2 answers
2k views

Find the unique MVUE

This question is from Robert Hogg's Introduction to Mathematical Statistics 6th Version problem 7.4.9 at page 388. Let $X_1,...,X_n$ be iid with pdf $f(x;\theta)=1/3\theta,-\theta<x<2\theta,$ ...
Deep North's user avatar
  • 4,746
9 votes
2 answers
2k views

Likelihood Function is Minimal Sufficient

What does it mean to say that "Likelihood Function is Minimal Sufficient"? Is this a general statement, or does it apply to only exponential family of distributions? I think I understand the concept ...
Cagdas Ozgenc's user avatar
9 votes
1 answer
413 views

Why is the EM algorithm well suited for exponential families?

I've been brushing up on the EM algorithm, and while I feel like I understand the basics, I keep seeing the claim made (e.g. here, here, among several others) that EM works particularly well for ...
MSR's user avatar
  • 91
9 votes
1 answer
1k views

The minimal sufficient statistic of $f(x) = e^{-(x-\theta)}e^{-e^{-(x-\theta)}}$

The Casella Berger (2002) solutions manual says that the minimal sufficient statistic for $$f(x) = e^{-(x-\theta)}e^{-e^{-(x-\theta)}}, \qquad x\in \mathbb{R}$$ are the order statistics $(X_{(1)},\...
Xiaomi's user avatar
  • 2,554
9 votes
1 answer
413 views

Sufficient statistics for $\mu_1 - \mu_2$

If $ X_1, ..., X_n$ is a random sample from $ X \sim N(\mu_1, \sigma^2)$ and $Y_1,..., Y_n$ is a random sample from $Y \sim N(\mu_2, \sigma^2),$ if the samples are independent and $ \sigma^2$ is known,...
Giiovanna's user avatar
  • 1,208
9 votes
0 answers
499 views

How do sufficiency statistics help in the interpretation of regression results?

One of the results why canonical link functions are widely used in GLMs is the existence of sufficiency statistics for the regression parameters, which in turn allow for: ... minimal sufficient ...
Alex's user avatar
  • 4,442
8 votes
1 answer
929 views

Does Basu's Theorem require minimal sufficiency?

Casella & Berger state Basu's Theorem (Th 6.2.24) as follows: If $T(X)$ is a complete and minimally sufficient statistic, then $T(X)$ is independent of every ancillary statistic. However, in ...
half-pass's user avatar
  • 3,750
8 votes
2 answers
5k views

Distribution of sum of order statistics

The question is from a problem I am trying to solve in Robert Hogg's introduction to Mathematical Statistics 6th version problem 7.2.9 in page 380. The problem is: We consider a random sample $X_1,...
Deep North's user avatar
  • 4,746
8 votes
3 answers
7k views

Is a minimal sufficient statistic also a complete statistic

I know that if a statistic is both sufficient and complete then it must also be minimal sufficient. But on the other hand, could I say a minimal sufficient statistic must also be a complete statistic?
zqzwxec11's user avatar
  • 101
8 votes
1 answer
5k views

Are complete statistics always sufficient?

I know that a complete sufficient statistic $T$ is such that 1) $T$ is sufficient for $\theta$, unknown parameter and 2) $T$ is complete. So, is it always the case? If the answer is not, what ...
PhDing's user avatar
  • 3,099
8 votes
2 answers
2k views

Puzzled by definition of sufficient statistics

I am learning about sufficient statistic from Mood, Graybill, and Boes's Introduction to the Theory of Statistics. I am slightly confused by the book's definition of a sufficient statistic for ...
Noppawee Apichonpongpan's user avatar
8 votes
3 answers
843 views

Sufficient Statistics - Relating the Intuition with the Mathematical Definition

I believe the heuristic definition of a Sufficient Statistic makes sense to me - when you take a sample in order to make an inference about the parameter related to the probability distribution, and ...
user523384's user avatar
8 votes
2 answers
1k views

Sufficient Statistic for non-exponential family distribution

Question: Let $X_1,X_2,\ldots,X_n$ be an iid sample from $N(\theta , 4 \theta^2 )$. I want to show that this model is not a member of the exponential family and to find a sufficient statistic for $\...
WeakLearner's user avatar
  • 1,491
8 votes
1 answer
7k views

What is exponential family criterion to test the sufficiency and completeness of an estimator?

I am struggling to understand the following result from Casella and Berger about sufficiency and completeness for exponential families: Let $X_{1},X_{2},...,X_{n}$ be iid observations from an ...
userNoOne's user avatar
  • 1,048
8 votes
1 answer
4k views

What is the meaning of the notation $P_\theta()$, where a probability has a subscript Greek letter?

What does theta subscript imply in e.g. this case: $$ P_\theta(T(x)=t) = 0 $$
Alex's user avatar
  • 91
8 votes
2 answers
945 views

Conditioning in the definition of sufficient statistics

Let $X_1,...,X_n$ be an i.i.d. sample with parameter $\theta$ and $T$ a statistics. The statistics is called sufficient if, given a value $t$, the distribution $P_{\theta}(X_1,..,X_n|T=t)$ does not ...
Thomas's user avatar
  • 910
8 votes
2 answers
3k views

How to show that a sufficient statistic is NOT minimal sufficient?

My homework problem is to give a counterexample where a certain statistic is not in general minimal sufficient. Irrespective of the details of finding a particular counterexample for this particular ...
Chill2Macht's user avatar
  • 6,309
8 votes
1 answer
2k views

Minimum dimension of sufficient statistics

Suppose that we have a parameter of $k$-dimensions. Say, for example, for $N(u,\theta)$ both unknown then the parameter is of two dimensions, and $n$ i.i.d. observations. Is it possible to find a ...
nobody's user avatar
  • 611
8 votes
1 answer
243 views

The distribution of a sufficient statistic

If I understand correctly, a distribution in the exponential family... $$\underline X\sim f_{\underline\theta}(\underline x) = exp\{\sum\limits_{i}\eta_i(\underline\theta)T_i(\underline x)-B(\...
f1r3br4nd's user avatar
  • 2,354
7 votes
2 answers
5k views

Sufficient Statistic for $\beta$ in OLS

I have the classical regression model $$y = \beta X + \epsilon$$ $$\epsilon \sim N(0, \sigma^2)$$ where $X$ is taken to be fixed (not random), and $\hat\beta$ is the OLS estimate for $\beta$. It is ...
Cagdas Ozgenc's user avatar
7 votes
3 answers
300 views

Likelihood function when $X\sim U(0,\theta)$

Let $X_1, ..., X_n$ be $i.i.d$ random variables, uniformly distributed over $(0,\theta)$. Derive the likelihood function given the sample $x_1, ..., x_n$. Answer The likelihood function is: \begin{...
AlexMe's user avatar
  • 571
7 votes
2 answers
629 views

Is there a difference between Bayesian and Classical sufficiency?

The title pretty much says it all. I wonder whether there is any difference in the way Bayesians understand sufficiency vs. the way orthodox statistics understands sufficiency, or are they equivalent? ...
Sebastian's user avatar
  • 3,094
7 votes
1 answer
795 views

Is the negative exponential distribution a member of the exponential family?

Please correct me if I am wrong. The general form of $k$-parameter exponential family is $$f(x;\boldsymbol{\theta}) = a(\boldsymbol{\theta})g(x) \exp\{\sum_{i=1}^{k}b(\boldsymbol{\theta}) R_i(x)\}$$ ...
Sheikh's user avatar
  • 398
7 votes
1 answer
2k views

Whether the minimal sufficient statistic is complete for a translated exponential distribution

Let $X_1, X_2..., X_n$ follows iid negative exponential distribution with pdf $$f(x) = \frac{1}{\theta^2} \: e^{-\frac{(x-\theta)}{\theta^2}} \: \: I_{(x>\theta)} $$ I have to show whether the ...
n1234's user avatar
  • 73
7 votes
1 answer
1k views

Relationship between completeness and sufficiency

hopefully this isn't a duplicate of another question (at least I didn't find one). Here is a question I have about completeness and sufficiency: Problem: Suppose $T(x)$ is complete sufficient for $\...
asdf's user avatar
  • 384
7 votes
1 answer
2k views

Likelihood ratio minimal sufficient

Consider a family of density functions $f(x|\theta)$ where the parameter space for $\theta$ is finite, that is, $\theta \in \{\theta_1, \cdots, \theta_p\}$. Assume that $\theta_p$ is such that $f(x|\...
elbarto's user avatar
  • 469
7 votes
1 answer
151 views

Why sample size is not a part of sufficient statistic?

Following simple example from Wikipedia's definition of sufficient statistic with Bernoulli distribution with parameter $\theta$, where sufficient statistic is a sum of successes $$T(X_n)=\sum_{i=1}^n ...
mikowai's user avatar
  • 118
7 votes
1 answer
426 views

Efficient Estimator from Insufficient Statistic

Suppose that I have a statistic $T(X)$, and I know for sure that it is not sufficient to estimate a parameter $\theta$. Is it still possible to have an estimator $\hat\theta(T(X))$ that is efficient (...
Cagdas Ozgenc's user avatar

1
2 3 4 5
10