31
votes
Accepted
Why do we not care about completeness, sufficiency of an estimator as much anymore?
We still care. However, a large part of statistics is now based on a data-driven approach where these concepts may not be essential or there are many other important concepts.
With computation power ...
- 1,472
24
votes
Accepted
Likelihood Function is Minimal Sufficient
Consider an observable data vector $\mathbf{x} = (x_1,...,x_n) \in \mathscr{X}$ with a joint-distribution that is indexed by the parameter $\theta \in \Theta$. It is possible to establish that "...
- 133k
23
votes
Accepted
How can a statistician who has the data for a non-normal distribution guess better than one who only has the mean?
For a uniform distribution between $0$ and $2 \mu$, the player who guesses the sample mean would do worse than one which guesses $\frac{3}{5} \max(x_i)$ (the sample maximum is a sufficient statistic ...
- 26.5k
19
votes
Accepted
Solution to German Tank Problem
Likelihood
Common problems in probability theory refer to the probability of observations $x_1, x_2, ... , x_n$ given a certain model and given the parameters (let's call them $\theta$) involved. For ...
- 86.5k
19
votes
Accepted
What is "Likelihood Principle"?
The concept of the likelihood principle (LP) is that the entire inference should be based on the likelihood function and solely on the likelihood function. Informally, the likelihood function is ...
- 108k
17
votes
Why do we not care about completeness, sufficiency of an estimator as much anymore?
We do care but usually either the issue is taken care of, or we're not making a specific distributional assumption with which we could apply those considerations.
Many of the usual estimators for ...
- 290k
16
votes
Accepted
Sufficient Statistic for $\beta$ in OLS
Sometimes the simplest way to look at sufficiency is by looking directly at the log-likelihood and using the factorisation theorem. For a linear regression model with Gaussian error term the log-...
- 133k
14
votes
Sufficient statistics for layman
Say you have a coin, and you don't know whether it's fair or not. In other words, it has probability $p$ of coming up heads ($H$) and $1 - p$ of coming up tails ($T$), and you don't know the value of $...
- 1,203
14
votes
Accepted
Mean is not a sufficient statistic for the normal distribution when variance is not known?
$\bar X$ is not a sufficient statistic because it does not contain all the information about $(\mu,\sigma^2)$, which is what it would mean for it to be sufficient.
However, $\bar X$ does contain all ...
- 46.8k
13
votes
Is there a standard measure of the sufficiency of a statistic?
Fisher's information associated with a statistic $T$ is the Fisher information associated with the distribution of that statistic
$$I_T(\theta) = \mathbb E_\theta\Big[\frac{\partial}{\partial \theta}\...
- 108k
12
votes
Accepted
Sufficient statistics for Uniform $(-\theta,\theta)$
Suppose we have a random sample $(X_1,X_2,\cdots,X_n)$ drawn from $\mathcal U(-\theta,\theta)$ distribution.
PDF of $X\sim\mathcal U(-\theta,\theta)$ is
$$f(x;\theta)=\frac{1}{2\theta}\mathbf1_{-\...
- 11.6k
12
votes
Accepted
What is the score function of two parameters?
The score for a multiple parameter problem (a vector parameter) is itself a vector. We need to take partial derivatives of the log likelihood with respect to each model parameter.
Let's consider an ...
- 2,937
12
votes
How can a statistician who has the data for a non-normal distribution guess better than one who only has the mean?
The sum of observations is not sufficient for estimating the mean of a uniform
population. The midrange has a smaller expectation of absolute error.
Approximation by simulation in R:
...
- 57.6k
11
votes
Jointly Complete Sufficient Statistics for Uniform$(a, b)$ Distributions
Let's take care of the routine calculus for you, so you can get to the heart of the problem and enjoy formulating a solution. It comes down to constructing rectangles as unions and differences of ...
- 334k
11
votes
How can a statistician who has the data for a non-normal distribution guess better than one who only has the mean?
It might be worth adding that while you can often do better for low-dimensional parametric families, you can't do better if the distribution is completely unknown (or completely unknown apart from ...
- 46.8k
10
votes
Does a sufficient statistic imply the existence of a conjugate prior?
If there exists a finite dimensional conjugate family,
$$\mathfrak F=\{\pi(\cdot|\alpha)\,;\ \alpha\in A\}$$
with $\dim(A)=d$, this means that, for any $\alpha\in A$, there exists a mapping $\tilde\...
- 108k
9
votes
Accepted
Sufficient statistic for bivariate or multivariate normal
As W. Huber tried to lead you to conclude, the sufficiency is a simple consequence of looking at the likelihood:
\begin{align*}
f(\mathbf{x}_1,\ldots,\mathbf{x}_n|\boldsymbol{\mu},\boldsymbol{\Sigma}) ...
- 108k
9
votes
Is there a difference between Bayesian and Classical sufficiency?
Here is one example of differentiation between classical and Bayesian statistics: when comparing two models $\mathcal{M}_1$ and $\mathcal{M}_2$, a statistic $S(\cdot)$ may be sufficient for both ...
- 108k
9
votes
Why is the weak likelihood principle not a theorem?
Fermat's Last Theorem is a proposition of Number Theory, so you'd want to prove it from Peano's axioms; the parallel postulate, of Euclidean geometry, so from Euclid's other four postulates: but the ...
- 31.6k
8
votes
Sufficiency of Sample Mean for Laplace Distribution
For on observation, the Laplace pdf is
$$f_X(x) = \dfrac 1 {2b} \exp(-\dfrac {|x-\mu|} b)$$
For multiple iid observations, the pdf is
$$f_\boldsymbol X(\boldsymbol x) = \dfrac 1 {(2b)^n} \exp(- \dfrac ...
- 1,327
8
votes
Accepted
Intuitive understanding of the Halmos-Savage theorem
A Technical Lemma
I'm not sure how intuitive this is, but the main technical result underlying your statement of the Halmos-Savage Theorem is the following:
Lemma.
Let $\mu$ be a $\sigma$-finite ...
- 4,107
8
votes
Accepted
Puzzled by the definition of sufficient statistics in Mood, Graybill, and Boes
All those interpretations seem to be a variation of expressing the same thing:
The independence of the distribution of the sample $X$ on a true parameter $\theta$ and the statistic $T$.
Which means ...
- 86.5k
8
votes
Accepted
Is there a difference between Bayesian and Classical sufficiency?
In addition to the other excellent answer: The question of equivalence between Bayesian (B-) sufficiency and Classical (F-sufficiency) is answered in the abstract by NOT. But, this is based on ...
- 82.8k
8
votes
Accepted
Sufficient statistic when $X\sim U(\theta,2 \theta)$
Regarding 1., note that interpretation of a sufficient statistic is:
"no other statistic that can be calculated from the same sample provides any additional information as to the value of the ...
- 16k
8
votes
Likelihood function when $X\sim U(0,\theta)$
You are correct that the third equality could use the event $\{\min(X_1, \ldots, X_n)\geq 0, \max(X_1, \ldots, X_n)\leq\theta\}$.
The more compact notation is used because for any value of $\theta$, $...
- 2,116
8
votes
How to prove any one-to-one function of minimal sufficient statistic is minimal sufficient?
By the measure-theoretic definition of sufficiency and minimal sufficiency, this statement is quite obvious. While the proof itself is just a one-liner, to comprehend underlying definitions needs a ...
- 21.2k
7
votes
When using the likelihood function, where does the indicator function come from?
Most families of distributions $f_\theta$ have a fixed support,
$$\text{supp}(f_\theta)=\{x\in\mathcal{X};\ f_\theta(x)>0\}$$
like the Normal or Binomial distributions, but some have a parameter ...
- 108k
7
votes
Accepted
Sufficient Statistic for non-exponential family distribution
First this is an exponential family (as shown by the above excerpt from Brown, 1986) since the density writes down as
$$\exp\{\Phi_1(\theta) S_1({\mathbf x})+\Phi_2(\theta) S_2({\mathbf x})-\Psi(\...
- 108k
7
votes
Accepted
Sufficient order statistics
Your confusion stems from the loose notations I presume: when writing
$$f(\mathbf{x}) = \prod_{i=1}^n f(x_i) = \prod_{i=1}^n f(x_{(i)}),$$
George Casella and Roger Berger first use $f$ for the density ...
- 108k
7
votes
Accepted
What is exponential family criterion to test the sufficiency and completeness of an estimator?
The significance of that line is that if you can verify that the parameter space contains an open set in $R^k$, you know, without any further work, that the sufficient statistics $T(X)$ are also ...
- 41.1k
Only top scored, non community-wiki answers of a minimum length are eligible