14
$\begingroup$

From Wikipedia:

The statistic $s$ is said to be complete for the distribution of $X$ if for every measurable function $g$ (which must be independent of parameter $θ$) the following implication holds: $$ \mathbb{E}_\theta[g(s(X))] = 0, \forallθ \text{ implies that }P_θ(g(s(X)) = 0) = 1, \forall θ. $$ The statistic $s$ is said to be boundedly complete if the implication holds for all bounded functions $g$.

I read and agree with Xi'an and phaneron that a complete statistic means that "there can only be one unbiased estimator based on it".

  1. But I don't understand what Wikipedia says at the beginning of the same article:

    In essence, it (completeness is a property of a statistic) is a condition which ensures that the parameters of the probability distribution representing the model can all be estimated on the basis of the statistic: it ensures that the distributions corresponding to different values of the parameters are distinct.

    • In what sense (and why) does completeness "ensures that the distributions corresponding to different values of the parameters are distinct"? is "the distributions" the distributions of a complete statistic?

    • In what sense (and why) does completeness "ensures that the parameters of the probability distribution representing the model can all be estimated on the basis of the statistic"?

  2. [Optional: What does "bounded completeness" mean, compared to completeness?]

$\endgroup$

marked as duplicate by kjetil b halvorsen, Michael Chernick, Peter Flom Feb 11 at 11:25

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • 3
    $\begingroup$ Check this other question: stats.stackexchange.com/questions/41881/… $\endgroup$ – Zen Mar 23 '13 at 23:33
  • $\begingroup$ @Zen:Thanks! Why do we need the "bounded completeness" then? $\endgroup$ – Tim Mar 23 '13 at 23:48
  • 4
    $\begingroup$ Both are technical (regularity) conditions that make more sense in the context of the proofs of the theorems in which they are involved. Hence, my advice would be to study the proofs of the Lehmann-Scheffé Theorem, Bahadur's Theorem and Basu's Theorem. $\endgroup$ – Zen Mar 24 '13 at 0:21
  • 2
    $\begingroup$ I am quite sceptical that completeness in itself implies parameter identifiability: start with a complete statistics for a family of distributions indexed by $\theta$ and add an extra and useless parameter $\eta$. Then the statistic remains complete. $\endgroup$ – Xi'an Oct 10 '14 at 19:49
  • $\begingroup$ See also my relevant answer here $\endgroup$ – kjetil b halvorsen Feb 7 at 10:00
15
$\begingroup$

This is a very good question and one I've struggled with for quite some time. Here's how I've decided to think about it:

Take the contrapositive of the definition as stated in Wikipedia (which doesn't change the logical meaning at all):

\begin{align} {\rm If}\quad &\neg\ \forall \theta\ P(g(T(x))=0)=1 \\ {\rm then}\quad &\neg\ \forall \theta\ E(g(T(x))) = 0 \end{align}

In other words, if there is a parameter value such that $g(T(x))$ is not almost surely $0$, then there is a parameter value such that the expected value of that statistic is not $0$.

Hmm. What does that even mean?

Let's ask what happens when $T(x)$ is NOT complete...

A statistic $T(x)$ that is NOT complete will have at least one parameter value such that $g(T(x))$ is not almost surely $0$ for that value, and yet it's expected value is $0$ for all parameter values (including this one).

In other words, there are values of $\theta$ for which $g(T(x))$ has a non-trivial distribution around it (it has some random variation in it), and yet the expected value of $g(T(x))$ is nonetheless always $0$--it doesn't budge, no matter how much $\theta$ is different.

A complete statistic, on the other have, will budge in it's expected value eventually if $g(T(x))$ is non-trivially distributed and centered at $0$ for some $\theta$.

Put another way, if we find a function $g(\cdot)$ where the expected value is zero for some $\theta$ value (say $\theta_0$) and it has a non-trivial distribution given that value of $\theta$, then there must be another value of $\theta$ out there (say, $\theta_1 \ne \theta_0$) that results in a different expectation for $g(T(x))$.

This means we can actually use this statistic for hypothesis testing and informative estimation in the context of an assumed distribution for our data. We want to be able to center it around a hypothesized value of $\theta$ and get it to have expectation 0 for that hypothesized value of $\theta$, but not for all other values of $\theta$. But if the statistic is not complete we may not be able to do this: we may be unable to reject any hypothesized values of $\theta$. But then we can't build confidence intervals and do statistical estimation.

$\endgroup$
  • 5
    $\begingroup$ Welcome to the site, & thanks for contributing this. I took the liberty of formatting your answer using the $\LaTeX$ markup our site affords. If you don't like it, roll it back with my apologies. (If you do like it, there is more information on formatting here.) Since you're new here, why not register your account & take our tour, which contains information for new users. $\endgroup$ – gung Jan 2 '15 at 1:14
11
$\begingroup$

Geometrically, completeness means something like this: if a vector $g(T)$ is orthogonal to the p.d.f. $f_\theta$ of $T$ for each $\theta$, $$\mathbb E_\theta g(T) = \langle g(T),f_\theta\rangle=0$$ then $g(T)=0$ i.e., the functions $f_\theta$ for varying $\theta$ span the whole space of functions of $T$. So in a way it would be more natural to say that

$\theta$ is complete for $T$

than what we do say,

$T$ is complete for $\theta$.

This way it is not so strange that a constant function would be "complete"!


Maybe an example helps.

Suppose $X$ and $Y$ are independent and identically distributed Bernoulli($\theta$) random variables taking values in $\{0,1\}$, and $Z=X-Y$. Then $Z$ is incomplete for $\theta$, because taking $g=\text{identity}$, $$\mathbb E_\theta(Z)=0$$ for all $0<\theta<1$, but nevertheless $\mathbb P_\theta(Z=0)\ne 1$.

$\endgroup$
  • $\begingroup$ This way shows the realtionsship to what "cxompleteness" means in Hilbert space settings ... $\endgroup$ – kjetil b halvorsen Feb 19 '16 at 12:32
3
$\begingroup$

I found this very helpful:

Definition: A statistic $T$ is called complete if $E_\theta[g(T)] = 0$ for all $\theta$ and some function $g$ implies that $P_\theta(g(T) = 0) = 1$ for all $\theta$.

Think of this as analog to vectors and whether or not the vectors {$v_1, \ldots , v_n$} form a complete set (=basis) of the vector space.

  • If they span the whole space, any $v$ can be written as a linear combination of these vectors: $v = \sum_j a_j \cdot v_j$.
  • Furthermore, if a vector $w$ is orthogonal to all $v_j$’s, then $w = 0$.

To make the connection with the completeness definition, let’s consider the case of a discrete probability distribution. We start by writing out the completeness condition $$ 0 = E_\theta[g(t)] = \sum_t g(t) \cdot P_\theta(T = t) = \sum_j g(t_j) \cdot P_\theta(T = t_j) = \begin{bmatrix} g(t_1)\\ g(t_2)\\ \ldots \\ g(t_n)\\ \end{bmatrix} \cdot \begin{bmatrix} p_\theta(t_1)\\ p_\theta(t_2)\\ \ldots \\ p_\theta(t_n)\\ \end{bmatrix} $$ for all $\theta$. Here we expressed the sum as a scalar product of two vectors $$(g(t_1), g(t_2),...)$$ and $$(p_\theta(t_1), p_\theta(t_2),...)$$, with $p_\theta(t_j) = P_\theta(T = t_j) \ne 0$ -- we consider only positive probabilities, because if $p(t_j) = 0$ this does not tell us anything about the function $g(t_j)$. Now we see the analogy to the orthogonality condition discussed above.

In principle it could be that the $g(t_j)$'s are non-zero, but that they sum to zero. However, as stated by Lhunt, this is only possible if

  • the probability vector $(p_\theta(t_1), p_\theta(t_2),...)$ does either not change at all if $\theta$ is varied,
  • or if it changes in a "simple way", e.g. it jumps from one value for all $j$'s to an other value for all $j$'s,
  • or if it changes in a "correlated way", which would be a nightmare to handle.

Thus, the cross-cancelation of terms is only possible if the probability distribution provides either a "boring set of basis vectors" or a nightmare.

In contrast, if the probability distribution provides a "sufficient rich set of basis vectors", the equation for the expectation value implies $g(t_j) = 0$ almost everywhere. By almost everywhere we mean, that there could be a set of probability zero, where $g(t_j) \ne 0$ -- e.g. in the case of a continuous probability distribution this could be a set of single points.

We also see that the terminology is somewhat misleading. It would be more accurate to call the family of distributions $p_\theta(\cdot)$ complete (rather than the statistic $T$) -- as stated in the original question. In any event, completeness means that the collection of distributions for all possible values of $\theta$ provides a sufficiently rich set of vectors.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.