Suppose that we have a parameter of $k$-dimensions. Say, for example, for $N(u,\theta)$ both unknown then the parameter is of two dimensions, and $n$ i.i.d. observations.

Is it possible to find a sufficient statistic that has dimension less than $k$? For example, for normal distribution the minimal sufficient statistics is proven to be (sample mean, sample variance) so it is not possible for normal distribution, but I am wondering if this holds in general.


Since sufficiency of fixed dimension only occurs in exponential families (Darmois-Pitman-Koopman lemma), apart from distributions with varying support like the Uniform, let us consider an exponential family with parameter $\theta$ and density [against a fixed dominating measure] $$f_\theta(x)=\exp\left\{\sum_{i=1}^k a_i(\theta) T_i(x) -\psi(\theta)\right\}$$ Assuming the functions $a_i$ are linearly independent on the maximal support of $\theta$ (namely, the range of $\theta$'s for which the density is integrable), the model associated with this density can be reparameterised in $\alpha=(\alpha_1,\ldots,\alpha_k)$ which varies in (at least) the parameter space $$A=\left\{\alpha=(\alpha_1,\ldots,\alpha_k);\,\exists\theta\in\Theta, \alpha_i=a_i(\theta) \right\}$$at least, since the parameter space can be naturally expanded to its natural limit $$A=\left\{\alpha=(\alpha_1,\ldots,\alpha_k);\,\displaystyle{\int \exp\left\{\sum_{i=1}^k \alpha_i T_i(x)\right\}\text{d}\lambda(x)}<\infty \right\}\,,$$and that the functions $T_i$ are also linearly independent over the support $\cal X$ of $X$, the statistic $$T=(T_1(X),\ldots,T_k(X)$$is sufficient and with density $$g_\alpha(t)=\exp\left\{\sum_{i=1}^k \alpha_i t_i-\mu(\alpha)\right\}$$ against the appropriate dominating measure. Therefore on the natural space of the exponential family (which may be larger than the original parameter space), the sufficient statistic is of the same dimension as the parameter. Even though the domain of variation of $T(x)$ can be constrained by non-linear relations, there is a sample size after which the dimensional constraint vanishes.

A completely different approach, avoiding exponential families, is provided by

Edward W. Barankin and Melvin Katz, Jr.
Sufficient Statistics of Minimal Dimension
Sankhyā: The Indian Journal of Statistics Vol. 21, No. 3/4 (Aug., 1959), pp. 217-246

and they show the following result

enter image description here

where $r$ is the dimension of the sufficient statistic $T$ and $\rho(x^0)$ is the (local) rank of the second derivative of the log-likelihood in $\theta$ and $x$ [the definition is a bit too intricate to be reproduced here].


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.