Skip to main content

Questions tagged [sufficient-statistics]

A sufficient statistic is a lower dimensional function of the data which contains all relevant information about a certain parameter in itself.

Filter by
Sorted by
Tagged with
23 votes
7 answers
3k views

Why does a sufficient statistic contain all the information needed to compute any estimate of the parameter?

I've just started studying statistics and I can't get an intuitive understanding of sufficiency. To be more precise I can't understand how to show that the following two paragraphs are equivalent: ...
3 votes
0 answers
110 views

Does 𝑓 : 𝑝 𝜃 ↦ 𝑝 𝑇 | 𝜃 being injective imply statistic T is sufficient? [closed]

Wikipedia says ... consider the map $f:p_{\theta }\mapsto p_{T|\theta }$ which takes each distribution on model parameter $\theta$ to its induced distribution on statistic $𝑇$. The statistic $T$ is ...
0 votes
0 answers
17 views

Bias and mean-squared error of an estimator for the parameter of an exponential family after applying Rao-Blackwell [closed]

From here: "Suppose $X_1, \dots, X_n$ are independent $\exp(\lambda)$ random variables, where $\lambda$ is the rate parameter...$S = \Sigma_{i = 1} ^ {n} X_i$ is a sufficient statistic...$\mathbb{...
31 votes
3 answers
4k views

Sufficient statistics for layman

Can someone please explain sufficient statistics in very basic terms? I come from an engineering background, and I have gone through a lot of stuff but failed to find an intuitive explanation.
5 votes
1 answer
59 views

Sufficient statistic for the family of PERT distributions?

A beta distribution is one of the form $$ \text{constant}\times x^{\alpha-1} (1-x)^{\beta-1} \, dx \quad \text{ for } 0<x<1. $$ According to this Wikipedia article, the family of "PERT ...
1 vote
0 answers
10 views

Sufficient statistic as iso-surfaces in the distribution density. Is it possible to generalise to multiple parameters?

For continuous distributions, there is a geometric intuition behind sufficient statistics that regards a multivariate probability density as several iso-surfaces. This works at least for cases where a ...
2 votes
0 answers
147 views

Bayesian definition of sufficient statistics [duplicate]

Some time ago I wrote a question about what I think/thought (up to my understanding) is an ambiguity of the common definition of sufficient statistics : Conditioning in the definition of sufficient ...
3 votes
1 answer
113 views

Sufficiency for Truncated Geometric

Here is a deviant of a question I feel like I have seen several times on truncated exponentials and similar distributions for finding sufficient statistics: Let $$\mathbb{P}(Y=y)=\theta^y(1-\theta)^{\...
3 votes
1 answer
231 views

Are these statistics sufficient?

Question (Casella and Berger 6.5): Let $X_1 \ldots X_n$ be independent random variables with pdfs: $f(x_i|\theta)= \begin{cases} \frac{1}{2i\theta}, & -i(\theta - 1)<x_i<i(\theta+1) \\ 0,...
4 votes
0 answers
120 views

Relating sufficient statistics to parameters

I'm studying sufficient statistics and I came across this problem: A dataset consists of independent triples $(W_i,Y_i,Z_i)$ of independent random variables with distributions as follows, $$ W_i \sim ...
1 vote
1 answer
61 views

Find a two dimensional sufficient statistic for $\theta$

Let $\{X_i\}_{i=1}^n$ be conditional independent given $\theta$ with distribution $$p_{X_i | \theta} (x |\theta) = \frac{1}{2i\theta}, \ -i\theta<x<i\theta.$$ Find a two dimensional sufficient ...
3 votes
2 answers
593 views

Need help understanding Sufficient Statistics and using the formal definition

Let $X = (X_1, X_2, . . . , X_n)$ be a random sample from $\rm Poisson(\theta)$. Use the factorization theorem to find a sufficient statistic $T(X)$ and then use the formal definition of sufficiency ...
8 votes
2 answers
643 views

Is there a difference between Bayesian and Classical sufficiency?

The title pretty much says it all. I wonder whether there is any difference in the way Bayesians understand sufficiency vs. the way orthodox statistics understands sufficiency, or are they equivalent? ...
8 votes
2 answers
2k views

Puzzled by the definition of sufficient statistics in Mood, Graybill, and Boes

I am learning about sufficient statistic from Mood, Graybill, and Boes's Introduction to the Theory of Statistics. I am slightly confused by the book's definition of a sufficient statistic for ...
3 votes
2 answers
128 views

Minimal sufficient statistics of increasing dimensionality (not equal to the number of observations)

Restricting the attention to the case of fixed parameters support, it's my understanding that (minimal) sufficient statistics of fixed dimensionality, i.e. a fixed number of of them, exists in, and ...
4 votes
2 answers
332 views

What is the goal of sufficient dimension reduction? Under what circumstances can it be achieved?

I have recently heard the term "sufficient dimension reduction" tossed around, although I have struggled to find material on the concept that I fully understand or that clearly explains why ...
5 votes
1 answer
971 views

What is the intuition behind the factorization theorem? (Sufficient statistics)

By the Fisher's factorization theorem, a statistics is a sufficient statistic if (and only if) the joint density, $$ f(x_1, x_2, x_3, \dots x_n; \theta) $$ can be factorized into two functions, $ g(s; ...
3 votes
1 answer
38 views

How does knowing the sign of the population correlation affect the sufficiency of its statistic?

As noted here, the sufficient statistic for the correlation under bivariate normality is Pearson's $r$, the maximum likelihood estimate of $\rho$. I suppose, however, this does not guarantee that $r$ ...
1 vote
0 answers
47 views

Find minimal sufficient statistic of this random sample with cursed support

Suppose $X_1,X_2,...,X_n$ is a i.i.d random sample with probability mass function $p(x_i,\theta)$ where $x_i \in \{\theta,\theta+1,\theta+2,...\}$ and $\theta \in \mathbb{R}$. I claim that minimal ...
3 votes
1 answer
137 views

What are sufficiency and parameterization invariance in the context of Maximum Likelihood estimation?

While looking at properties of MLE, the article mentions 4 things below: sufficiency (complete information about the parameter of interest contained in its MLE estimator) consistency efficiency ...
3 votes
1 answer
362 views

Is the Sufficiency Principle an axiom?

Sufficiency Principle as defined in Casella: Where Sufficient Statistic is defined as: Question: Is the Sufficiency Principle an axiom? My thoughts and research so far: I'm uncertain if the ...
12 votes
1 answer
2k views

What is "Likelihood Principle"?

While I was studying "Bayesian Inference", I happen to encounter the term, "Likelihood Principle" but I don't really get the meaning of it. I assume it is connected to "...
2 votes
1 answer
37 views

Why does the sufficient statistic for the bivariate normal not imply a sufficient statistic for the correlation under bivariate normality?

This question links to a document by Jon Wellner that defines the sufficient statistic for the multivariate normal (p. 7, Example 2.7). The result follows from the factorization theorem and is proven ...
2 votes
0 answers
39 views

Reference request for the existence of minimal sufficient statistics

I'd like a recent paper or book that shows in what conditions we can guarantee the existence of a minimal sufficient statistic. I know the paper "Sufficiency and Statistical Decision Functions&...
6 votes
3 answers
131 views

Is Pitman-Koopman-Darmois Theorem valid for discrete random variables?

I am interested in the Pitman-Koopman-Darmois theorem. I'm having a hard time finding a simple rigorous version of this theorem as I struggle finding sources. This helpful post provides three sources ...
0 votes
0 answers
70 views

Formal definition of sufficient statistic

Let $(\Omega_X,\mathcal{F}_X)$ and $(\Omega _T,\mathcal{F}_T)$ be measurable spaces. Let $\mathfrak{M}$ be a family of probability measures on $(\Omega_X,\mathcal{F}_X)$. Let $X:\Omega\to \Omega _X$ ...
0 votes
0 answers
45 views

Sufficient Statistic for Truncated Normal

I am doing exercise 3.18 of "The Bayesian Choice": Give a sufficient statistic associated with a sample $x_1,...,x_n$ from a truncated normal distribution $$ f (x|\theta) \propto \exp(-(x ...
1 vote
1 answer
2k views

Jointly Sufficient Statistic Question

So here is a problem I have been working on: Suppose that survival time $X$ has a lognormal distribution with parameters $\mu$ and $\theta$ (which are the mean and standard deviation of $\log(X)$, not ...
2 votes
2 answers
363 views

Sufficient statistic $\sum_{j=1}^{n} |x_{j}|$ for laplace distribution

Let be $X_{1},\ldots , X_{n}$ random variables independent and identically distributed with density function: $$ f_{\theta}(x)=\dfrac{1}{2}e^{-|x-\mu|}, \quad x,\mu \in \mathbb{R} $$ Find the joint ...
3 votes
3 answers
805 views

How to prove any one-to-one function of minimal sufficient statistic is minimal sufficient?

So I want to prove that any one-to-one function of minimal sufficient statistic is also minimal sufficient. Here is my proof: Let $T$ be a minimal sufficient statistic and $f$ is a one-to-one function ...
7 votes
1 answer
1k views

Relationship between completeness and sufficiency

hopefully this isn't a duplicate of another question (at least I didn't find one). Here is a question I have about completeness and sufficiency: Problem: Suppose $T(x)$ is complete sufficient for $\...
1 vote
0 answers
38 views

How does reparametrization of the Fisher information matrix change the variance expression for the sufficient statistics?

If I have an exponential family distribution of the form $$p_{\theta}(x) = e^{\theta^T\cdot t(x) - \psi(\theta)},$$ where $\theta$ is a vector of parameters, $t(x)$ is a vector of sufficient ...
3 votes
1 answer
55 views

Constructing likelihood for a statistical functional

I'm a bit confused about something that I've seen in several engineering papers. Suppose we have a random variable $X$ with unknown distribution $p(x)$. We're interested in the value of a ...
0 votes
0 answers
60 views

Usage of Sufficient statistic for a Gamma distribution

I need some help to understand how to utilize sufficient statistic from a data. Suppose I observe some random process that produces $x\in X$, where all elements have a gamma distribution. As far as I ...
5 votes
3 answers
161 views

Likelihood principle and inference

I've been reading Casella and Berger's Statistical Inference. In section 6.3 the author stated the likelihood principle: if the likelihood functions from two samples are proportional, then the ...
0 votes
1 answer
277 views

Are Skewness and Kurtosis Sufficient Statistics?

I would like to prove that Skewness and Kurtosis are sufficient statistics for gaussian distribution. Later on I will try to prove on loglogistic distribution. Do you have any idea how to do that?
9 votes
1 answer
413 views

Sufficient statistics for $\mu_1 - \mu_2$

If $ X_1, ..., X_n$ is a random sample from $ X \sim N(\mu_1, \sigma^2)$ and $Y_1,..., Y_n$ is a random sample from $Y \sim N(\mu_2, \sigma^2),$ if the samples are independent and $ \sigma^2$ is known,...
1 vote
1 answer
41 views

FInding a complete and sufficient statistic

I am attempting to learn how to find a complete and sufficient statistic. So, I am working on this problem for class: Let $X_1, \cdot\cdot\cdot,X_n$ be a random sample from the pdf $f(x_i|u)=e^{-(x-\...
7 votes
1 answer
2k views

Whether the minimal sufficient statistic is complete for a translated exponential distribution

Let $X_1, X_2..., X_n$ follows iid negative exponential distribution with pdf $$f(x) = \frac{1}{\theta^2} \: e^{-\frac{(x-\theta)}{\theta^2}} \: \: I_{(x>\theta)} $$ I have to show whether the ...
6 votes
2 answers
615 views

Producing samples from exponential family conditional on minimal sufficient statistic

Suppose I have a distribution which belongs to an exponential family, of the form $$p(x) = \frac{\exp(-\sum_k \eta_k T_k(x))}{Z},$$ where $T_k(x)$ are a fixed set of sufficient statistics, $\eta_k$ ...
3 votes
1 answer
174 views

Karlin-Rubin theorem: relationship between test statistic having the MLR property vs being sufficient

Let's suppose we are trying to compare two hypotheses for a single parameter $\theta$. The null hypothesis $H_0$ is that $\theta = \theta_0$, and the alternative is that $\theta ≥ \theta_0$. The ...
10 votes
2 answers
2k views

Find the unique MVUE

This question is from Robert Hogg's Introduction to Mathematical Statistics 6th Version problem 7.4.9 at page 388. Let $X_1,...,X_n$ be iid with pdf $f(x;\theta)=1/3\theta,-\theta<x<2\theta,$ ...
1 vote
2 answers
354 views

Showing that $X_{(1)}$ is sufficient for shifted exponential distribution

If the pdf of a random sample is $f(x)=e^{-(x-θ)}$ where $x \geq θ$, Show that $T=X_{(1)}$ is a sufficient statistic for $θ$. Can one show that $T$ is a sufficient statistic for $θ$ in the following ...
0 votes
0 answers
217 views

Rao blackwell theorem but the unbiased estimator is a function of the sufficient statistic

The Rao-Blackwell Theorem states the following: Let $T(\mathbf X)$ be a sufficient statistic for the statistical model $(S, \{f_{\theta}: \theta \in \Theta\})$ and $\hat \theta(\mathbf X)$ be and ...
4 votes
2 answers
496 views

Why is the weak likelihood principle not a theorem?

The weak likelihood principle (WLP) has been summarized as: If a sufficient statistic computed on two different samples has the same value on each sample, then the two samples contain the same ...
0 votes
1 answer
118 views

Unbiased estimator for parameter of random variables following a uniform distribution [duplicate]

Suppose $X_i$ are i.i.d. and have density $f_\theta(x) = \frac{1}{\theta}$ if $x \in (\theta, 2\theta)$ for positive $\theta$. $(\min_iX_i, \max_iX_i)$ is a sufficient statistic for $\theta$? To ...
1 vote
1 answer
2k views

Sufficient statistics for bernoulli distribution

Let $Y_1, \ldots, Y_n $ be a random sample of size $n$ where each $Y_i \sim \textrm{Bernoulli}(p), $ and let $Y = \sum Y_i $ for $i = 1, \ldots, n.$ The estimator is $W= (Y+1)/(n+2). $ Is the ...
8 votes
2 answers
3k views

How to show that a sufficient statistic is NOT minimal sufficient?

My homework problem is to give a counterexample where a certain statistic is not in general minimal sufficient. Irrespective of the details of finding a particular counterexample for this particular ...
2 votes
1 answer
134 views

Completeness of Gamma family

Let $X_1,...,X_n$ has a Gamma$(\alpha,\alpha)$ distribution. Find the minimal sufficient statistics. Is this a complete family? My attempt: I found the Minimal sufficient statistics is $T(x)=(\...
0 votes
1 answer
275 views

Sufficient statistic under transformation

I just read a weird question when I'm learning sufficient statistic myself. It says whether the sufficient statistic of variance and standard deviation are the same under normal distribution with ...

1
2 3 4 5
10