Skip to main content
31 votes
Accepted

Why do we not care about completeness, sufficiency of an estimator as much anymore?

We still care. However, a large part of statistics is now based on a data-driven approach where these concepts may not be essential or there are many other important concepts. With computation power ...
DanielTheRocketMan's user avatar
23 votes
Accepted

Likelihood Function is Minimal Sufficient

Consider an observable data vector $\mathbf{x} = (x_1,...,x_n) \in \mathscr{X}$ with a joint-distribution that is indexed by the parameter $\theta \in \Theta$. It is possible to establish that "...
Ben's user avatar
  • 128k
23 votes
Accepted

How can a statistician who has the data for a non-normal distribution guess better than one who only has the mean?

For a uniform distribution between $0$ and $2 \mu$, the player who guesses the sample mean would do worse than one which guesses $\frac{3}{5} \max(x_i)$ (the sample maximum is a sufficient statistic ...
shimao's user avatar
  • 26.3k
19 votes
Accepted

What is "Likelihood Principle"?

The concept of the likelihood principle (LP) is that the entire inference should be based on the likelihood function and solely on the likelihood function. Informally, the likelihood function is ...
Xi'an's user avatar
  • 106k
18 votes
Accepted

Solution to German Tank Problem

Likelihood Common problems in probability theory refer to the probability of observations $x_1, x_2, ... , x_n$ given a certain model and given the parameters (let's call them $\theta$) involved. For ...
Sextus Empiricus's user avatar
17 votes

Why do we not care about completeness, sufficiency of an estimator as much anymore?

We do care but usually either the issue is taken care of, or we're not making a specific distributional assumption with which we could apply those considerations. Many of the usual estimators for ...
Glen_b's user avatar
  • 285k
14 votes

Sufficient statistics for layman

Say you have a coin, and you don't know whether it's fair or not. In other words, it has probability $p$ of coming up heads ($H$) and $1 - p$ of coming up tails ($T$), and you don't know the value of $...
Denziloe's user avatar
  • 1,143
14 votes
Accepted

Sufficient Statistic for $\beta$ in OLS

Sometimes the simplest way to look at sufficiency is by looking directly at the log-likelihood and using the factorisation theorem. For a linear regression model with Gaussian error term the log-...
Ben's user avatar
  • 128k
14 votes
Accepted

Mean is not a sufficient statistic for the normal distribution when variance is not known?

$\bar X$ is not a sufficient statistic because it does not contain all the information about $(\mu,\sigma^2)$, which is what it would mean for it to be sufficient. However, $\bar X$ does contain all ...
Thomas Lumley's user avatar
13 votes

Is there a standard measure of the sufficiency of a statistic?

Fisher's information associated with a statistic $T$ is the Fisher information associated with the distribution of that statistic $$I_T(\theta) = \mathbb E_\theta\Big[\frac{\partial}{\partial \theta}\...
Xi'an's user avatar
  • 106k
12 votes
Accepted

What is the score function of two parameters?

The score for a multiple parameter problem (a vector parameter) is itself a vector. We need to take partial derivatives of the log likelihood with respect to each model parameter. Let's consider an ...
jcken's user avatar
  • 2,917
12 votes

How can a statistician who has the data for a non-normal distribution guess better than one who only has the mean?

The sum of observations is not sufficient for estimating the mean of a uniform population. The midrange has a smaller expectation of absolute error. Approximation by simulation in R: ...
BruceET's user avatar
  • 56.8k
11 votes

Jointly Complete Sufficient Statistics for Uniform$(a, b)$ Distributions

Let's take care of the routine calculus for you, so you can get to the heart of the problem and enjoy formulating a solution. It comes down to constructing rectangles as unions and differences of ...
whuber's user avatar
  • 328k
11 votes
Accepted

Sufficient statistics for Uniform $(-\theta,\theta)$

Suppose we have a random sample $(X_1,X_2,\cdots,X_n)$ drawn from $\mathcal U(-\theta,\theta)$ distribution. PDF of $X\sim\mathcal U(-\theta,\theta)$ is $$f(x;\theta)=\frac{1}{2\theta}\mathbf1_{-\...
StubbornAtom's user avatar
  • 11.4k
11 votes

How can a statistician who has the data for a non-normal distribution guess better than one who only has the mean?

It might be worth adding that while you can often do better for low-dimensional parametric families, you can't do better if the distribution is completely unknown (or completely unknown apart from ...
Thomas Lumley's user avatar
10 votes

Does a sufficient statistic imply the existence of a conjugate prior?

If there exists a finite dimensional conjugate family, $$\mathfrak F=\{\pi(\cdot|\alpha)\,;\ \alpha\in A\}$$ with $\dim(A)=d$, this means that, for any $\alpha\in A$, there exists a mapping $\tilde\...
Xi'an's user avatar
  • 106k
9 votes
Accepted

Sufficient statistic for bivariate or multivariate normal

As W. Huber tried to lead you to conclude, the sufficiency is a simple consequence of looking at the likelihood: \begin{align*} f(\mathbf{x}_1,\ldots,\mathbf{x}_n|\boldsymbol{\mu},\boldsymbol{\Sigma}) ...
Xi'an's user avatar
  • 106k
9 votes

Is there a difference between Bayesian and Classical sufficiency?

Here is one example of differentiation between classical and Bayesian statistics: when comparing two models $\mathcal{M}_1$ and $\mathcal{M}_2$, a statistic $S(\cdot)$ may be sufficient for both ...
Xi'an's user avatar
  • 106k
9 votes

Why is the weak likelihood principle not a theorem?

Fermat's Last Theorem is a proposition of Number Theory, so you'd want to prove it from Peano's axioms; the parallel postulate, of Euclidean geometry, so from Euclid's other four postulates: but the ...
Scortchi - Reinstate Monica's user avatar
8 votes

Sufficiency of Sample Mean for Laplace Distribution

For on observation, the Laplace pdf is $$f_X(x) = \dfrac 1 {2b} \exp(-\dfrac {|x-\mu|} b)$$ For multiple iid observations, the pdf is $$f_\boldsymbol X(\boldsymbol x) = \dfrac 1 {(2b)^n} \exp(- \dfrac ...
jjet's user avatar
  • 1,287
8 votes
Accepted

Intuitive understanding of the Halmos-Savage theorem

A Technical Lemma I'm not sure how intuitive this is, but the main technical result underlying your statement of the Halmos-Savage Theorem is the following: Lemma. Let $\mu$ be a $\sigma$-finite ...
Artem Mavrin's user avatar
  • 4,067
8 votes
Accepted

Puzzled by the definition of sufficient statistics in Mood, Graybill, and Boes

All those interpretations seem to be a variation of expressing the same thing: The independence of the distribution of the sample $X$ on a true parameter $\theta$ and the statistic $T$. Which means ...
Sextus Empiricus's user avatar
8 votes
Accepted

Is there a difference between Bayesian and Classical sufficiency?

In addition to the other excellent answer: The question of equivalence between Bayesian (B-) sufficiency and Classical (F-sufficiency) is answered in the abstract by NOT. But, this is based on ...
kjetil b halvorsen's user avatar
8 votes
Accepted

Sufficient statistic when $X\sim U(\theta,2 \theta)$

Regarding 1., note that interpretation of a sufficient statistic is: "no other statistic that can be calculated from the same sample provides any additional information as to the value of the ...
Greenparker's user avatar
  • 15.7k
8 votes

Likelihood function when $X\sim U(0,\theta)$

You are correct that the third equality could use the event $\{\min(X_1, \ldots, X_n)\geq 0, \max(X_1, \ldots, X_n)\leq\theta\}$. The more compact notation is used because for any value of $\theta$, $...
Robin Ryder's user avatar
  • 2,096
7 votes

Why does a sufficient statistic contain all the information needed to compute any estimate of the parameter?

As I was studying about sufficiency I came across your question because I also wanted to understand the intuition about From what I've gathered this is what I come up with (let me know what you think, ...
ogustavo's user avatar
  • 676
7 votes
Accepted

Sufficient Statistic for non-exponential family distribution

First this is an exponential family (as shown by the above excerpt from Brown, 1986) since the density writes down as $$\exp\{\Phi_1(\theta) S_1({\mathbf x})+\Phi_2(\theta) S_2({\mathbf x})-\Psi(\...
Xi'an's user avatar
  • 106k
7 votes

When using the likelihood function, where does the indicator function come from?

Most families of distributions $f_\theta$ have a fixed support, $$\text{supp}(f_\theta)=\{x\in\mathcal{X};\ f_\theta(x)>0\}$$ like the Normal or Binomial distributions, but some have a parameter ...
Xi'an's user avatar
  • 106k
7 votes
Accepted

Sufficient order statistics

Your confusion stems from the loose notations I presume: when writing $$f(\mathbf{x}) = \prod_{i=1}^n f(x_i) = \prod_{i=1}^n f(x_{(i)}),$$ George Casella and Roger Berger first use $f$ for the density ...
Xi'an's user avatar
  • 106k
7 votes

Are unbiased efficient estimators stochastically dominant over other (median) unbiased estimators?

Here is an experiment in a non-standard case, the location Cauchy problem, where non-standard means that there is no uniformly best unbiased estimator. Let us consider $(X_1,\ldots,X_N)$ a sample from ...
Xi'an's user avatar
  • 106k

Only top scored, non community-wiki answers of a minimum length are eligible