I have these conjectures that I cannot quite prove (unless I impose another regularity condition of parameter-independent support for distribution, in which case, the conjectures are trivially true --- I think).

Would appreciate 1) confirmation (w/ proof ideally), or 2) rejection (w/ counter example ideally).


Suppose $X_1, ..., X_n$ are IID scalar random variables with PDF $f(x; \theta_1, ..., \theta_k)$, where $k < n$ and $\theta_1, ..., \theta_k$ are well-identified without any constraints among them (I can make this technically precise if necessary). Suppose there are $s$ statistics, $T_1, ..., T_s$, each a scalar, that are jointly sufficient. Then,

A) $s \ge k$;

B) If $s = k$, then the sufficient statistics are minimal;

C) If $s = k$, then the sufficient statistics are complete.

End of Conjectures

Notes: If I further assume that the support of $f(x)$ is independent of $(\theta_1, ..., \theta_k)$, it is my understanding that there is a theorem that shows that $f(x)$ must be from the exponential family and all the conjectures above are true.


Nice conjectures you got there; shame if something were to happen to them

(A) is false for all $k>1$ for the simple reason that any vector of real numbers $T_1,...,T_s$ can be reduced to a single real number without loss of information (i.e., $\mathbb{R}^s$ is cardinally equivalent to $\mathbb{R}$; see e.g., here). For example (though there are many other mappings you could use), write the digits of the statistics when represented as decimal numbers as:

$$\begin{align} T_1 &= \cdots d_{1,3} \ d_{1,2} \ d_{1,1} \ d_{1,0} \cdot d_{1,-1} \ d_{1,-2} \ d_{1,-3} \cdots \\[6pt] T_2 &= \cdots d_{2,3} \ d_{2,2} \ d_{2,1} \ d_{2,0} \cdot d_{2,-1} \ d_{2,-2} \ d_{1,-3} \cdots \\[6pt] &\ \ \vdots \\[6pt] T_s &= \cdots d_{s,3} \ d_{s,2} \ d_{s,1} \ d_{s,0} \cdot d_{s,-1} \ d_{s,-2} \ d_{1,-3} \cdots \\[6pt] \end{align}$$

and then agglomerate these into a single real number:

$$T_* \equiv \cdots d_{1,1} \cdots d_{s,1} \ d_{1,0} \cdots d_{s,0} \cdot d_{1,-1} \cdots d_{s,-1} \ d_{1,-2} \cdots d_{s,-2} \cdots$$

It is simple to reverse this process to create a mapping $T_* \mapsto (T_1,...,T_s)$, which means that $T_* \in \mathbb{R}$ is also a sufficient statistic. Consequently, for any $k>1$ if $T_1,...,T_s$ is sufficient then we can formulate the sufficient statistic $T_*$ with length $s_* = 1 < k$.

(B) is false for all $k>1$ as a corollary of the above. Reduce the sufficient statistic to $T_*$ and then add useless statistics $T_{*2},...,T_{*k}$. This then gives you a vector $T_*,T_{*2},...,T_{*k}$ with length $k$ that is not minimal sufficient (since the latter elements contribute nothing to sufficiency). More generally, sufficiency of a statistic does not imply minimal sufficiency, even if the statistic has the same dimension as the parameter vector.

(C) is false by virtue of the fact that you can construct a sufficient non-complete statistic for the dimension $k=1$. For $k>1$ you can also apply the above to construct a sufficient statistic that is non-complete.

  • 2
    $\begingroup$ Thanks Ben. I actually saw this math result (cardinal equivalency) before, but failed to think of it when trying to "prove" the conjectures. Thanks for pointing it out. Now I don't need to bother. I knew these all but certain could not be true (otherwise, someone would have long ago done it). So indeed, the number of scalars involved in the sufficient statistics, without context, is arbitrary. It could be as few as one and as big as you like (even for complete sufficient stats). Conjecture (C) implies (B). So if (B) falls, (C) cannot stand. $\endgroup$ Jul 28 '21 at 15:33
  • 1
    $\begingroup$ Yes, unfortunately when dealing with real vectors, if we don't have any restriction on structure we can expand and contract them to any length $1 \leqslant \ell < \infty$ with the same information. Results somewhat like your conjectures sometimes hold when we restrict structure heavily (e.g., dealing with linear equations in unknowns), so the next useful step might be to think about whether there is any restriction you can impose on the statistics to give those results. $\endgroup$
    – Ben
    Jul 28 '21 at 22:28
  • 1
    $\begingroup$ I know there may be ways to repair such conjectures ... Barndoff/Pedersen (1968) paper got around your "trick" by requiring the sufficient stats be continuous mapping from the raw data, for example. But I will leave such attempts to the theoretical statisticians. I am already too far off field. $\endgroup$ Jul 29 '21 at 1:40

$U[0,\theta]$ is a counterexample to C. $T_1=\max_i X_i$ is sufficient for $\theta$ but it is not complete because $$E\left[\frac{n+1}{n}T_1\right]=\theta=E[2\bar X]$$

(I believe A and B are true.)

  • 1
    $\begingroup$ Thanks Thomas. "Theory of Point Estimation" by Lehmann and Casella, Example 6.23 proves that $T_1$ is complete sufficient. Perhaps you were thinking about examples like $U(0.5\theta, 1.5\theta)$ where the minimum sufficient stats, (min, max), are not complete? $\endgroup$ Jul 28 '21 at 14:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.