4
$\begingroup$

Suppose we have a random vector $\mathbf{X} = (X_1,\ldots,X_n)$ or sample with pdf or pmf $f(\mathbf{x};\theta)$, where $\theta \in \Theta$ is an unknown parameter (or vector of parameters), whose specification completely determines $f(\mathbf{x};\theta)$.

As far as I understand, a statistic is just a a random variable $T=T(\mathbf{x})$ that is a function (that does not depend on $\theta$) of $\mathbf{X}$. That's not to say the distribution of $T$ doesn't depend on $\theta$ of course, just that the function that relates $\mathbf{X}$ and $T$ does not.

Now, usually from what I've seen, the statistic $T$ is defined to be a sufficient statistic if the distribution of $\mathbf{X}$ conditional on $T=t$ does not depend on $\theta$.

Now, my issue with this definition (which I don't understand how is it not addressed anywhere I've looked) is that if the support of $T$ depends on $\theta$, then we can't really pick a $t$ in the support of $T$ and then claim that the conditional distribution does not depend on $\theta$, because that same conditional distribution may be ill-defined for some values of $\theta$ for which $t$ is not in the support of $T$, right?

I kinda came up with my own definition which I'm hoping someone could comment on to see if this is what's more formally meant by that usual definition:

A statistic $T(\mathbf{X})$ is sufficient for $\theta$ if there exists a non-negative function $C:\mathbb{R}^n\times \mathbb{R} \to \mathbb{R}$ that does not depend on $\theta$ and such that $$p(\mathbf{x},t;\theta) = C(\mathbf{x},t) \cdot q(t;\theta)$$ for all $\mathbf{x} \in \mathbb{R}^n, t\in \mathbb{R}, \theta \in \Theta$, where $p$ and $q$ denote the joint pdf/pmf of $(\mathbf{X},T(\mathbf{X}))$ and the pdf/pmf of $T(\mathbf{X})$, respectively.

Improve this question
$\endgroup$
9
  • 1
    $\begingroup$ (a) You're welcome to invent any kind of definition you want, but a definition by itself is without any utility. What matters is what you can use your definition for. Yours does not deserve to be called a "sufficient statistic" until you can show that all--or at least most--of the important properties and theorems enjoyed by sufficient statistics also hold for your definition. (b) I cannot make sense of the paragraph about "my issue." Could you elaborate on what you think the problem is? Perhaps offer a simple example? $\endgroup$
    – whuber ♦
    Oct 2, 2016 at 22:07
  • $\begingroup$ Sure, for instance, consider $X_1,\ldots,X_n$ to be iid variables following a uniform distribution on the interval $[0,\theta]$. Now, if we consider the statistic $T(X_1,\ldots,X_n) = \sum_{i=1}^n X_i$, then it's clear that the support of $T$ is $[0,n\theta]$, which clearly depends on $\theta$. Now, if I pick any $t$, even if $t>0$, then how can I claim the conditional distribution of $\mathbf{X}=(X_1,\ldots,X_n)$ conditional on $T=t$ is a constant function of $\theta$, if that conditional distribution isn't even defined for, say $\theta = \frac{t}{2n}$? $\endgroup$
    – user45453
    Oct 2, 2016 at 22:29
  • 1
    $\begingroup$ Maybe you should consult the Wikipedia article. $\endgroup$
    – whuber ♦
    Oct 2, 2016 at 22:44
  • 1
    $\begingroup$ If you are interested, I found the answer to my question I suppose. The more rigorous definition (in the measure theory sense) does address the issue I mentioned by basically imposing that it's not the conditional distribution of $\mathbf{X}|T(\mathbf{X})=t$ itself that needs to be independent of $\theta$, but what they call a version of it. Anyways, here's a link on an article that rigorously defines it this way (page 232, definition 5). $\endgroup$
    – user45453
    Oct 2, 2016 at 23:44
  • 1
    $\begingroup$ @user45453: You could write those details up and then answer your Q yourself $\endgroup$
    – kjetil b halvorsen ♦
    Jan 26, 2019 at 4:27

1 Answer 1

Reset to default
2
$\begingroup$

The most frequently seen example of this is $X_1,\ldots,X_n\sim\operatorname{iid} \operatorname{Uniform}(0,\theta).$ It is said that the conditional distribution of $(X_1,\ldots,X_n)$ given $\max\{X_1,\ldots,X_n\}$ does not depend on $\theta.$

But to be precise, it ought to be said that it does not depend on which of the values of $\theta$ consistent with the observed value of $\max\{X_1,\ldots,X_n\}$ is the true value. Then one would take that to be the definition of sufficiency of the maximum.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.