4
$\begingroup$

I have a question that relates to this post:

Can a statistic depend on a parameter?

But on it, the discussion focuses much on the t-statistic given as an example by the question asker. My doubt in a broader sense is that:

Let ${X_1, ..., X_n}$ be a random sample of size $n$ from a population. $T(x_1, ..., x_n)$ is a real-valued function. The random-variable $Y = T(X_1, ..., X_n)$ is called a statistic.

The statistic can't be a function of any parameter. But the random sample ${X_1, ..., X_n}$ depends on some parameter $\theta$. So, if the statistic is a function of the random sample, and the random sample is a function of a parameter, doesn't that make the (random) statistics a function of the parameter as well?

I understand that when we are calculating a t-statistic, say, we aren't using the real parameter of the population anywhere. But we're using a sample mean. And this sample mean is dependent on the populational mean, ain't it? So the (random) statistic depends in some sense of the populational mean.

Then, $T(\textbf{X}) = T(\textbf{X}(\theta))$. But that goes against the fact that the statistic can't be a function of any parameter. That just doesn't enter my head when I think of the random counterpart of the statistic.

There must be something wrong with my line of thought but I just can't find it. Any thoughts?

$\endgroup$
4
$\begingroup$

$\require{mediawiki-texvc}$Let $T=T(X)=T(X_1,X_2, \dotsc, X_n)$ be a statistic, and assume we have some statistical model for the random variable $X$ (the data), say that $X$ is distributed according to the distribution $f(x;\theta)$, $f$ is then a model function (often a density or probability mass function) which is known only up to the parameter $\theta$, which is unknown.

Then the statistic $T$ has a distribution which depend upon the unknown parameter $\theta$, but $T$, as a function of the data $X$, do not depend upon $\theta$. That only says that you can calculate the realized value of $T$, from some observed data, without knowing the value of the parameter $\theta$. That is good, because you do not know $\theta$, so if you needed $\theta$ to calculate $T$, you would not be able to calculate $T$. That would be bad, because you could not even start your statistical analysis!

But, still the distribution of $T$ depends upon the value of $\theta$. That is good, because it means that observing the realized value of $T$ you can guess something about $\theta$, maybe calculate a confidence interval for $\theta$. If the distribution of $T$ was the same for all possible values of $\theta^\P$, then observing the value of $T$ would not teach us anything about $\theta$!

So, this boils down to: You must distinguish between $T$ as a function of the data, and the distribution of the random variable $T(X)$. The first one do not depend upon $\theta$, the second one does.

$\P$: Such a statistic is called ancillary. It might be useful, just not directly, alone for inference about $\theta$.

$\endgroup$
1
$\begingroup$

While the other answers (so far) are quite to the point and valid, I would like to add another direction to the discussion that relates to both fiducial inference (Fisher's pet theory) and a form of sampling called "perfect sampling" (or "sampling from the past").

Since a random variable is a measurable function from a (probability) space $(\Omega,\mathbb{P})$ to $\mathbb{R}$ (or $\mathbb{R}^n$), $X:\Omega\to\mathbb{R}$, the function may itself depend on a parameter $\theta$ if its distribution depends on $\theta$, in the sense that $X=\Psi(\omega,\theta)$. For instance, if $F_\theta$ denotes the cdf of $X$, we can write $X=F_\theta^{-1}(U)$ where $U$ is a uniform $\mathcal{U}(0,1)$ random variable. In this sense, $X$ (and the sample $(X_1,\ldots,X_n)$ as well) can be written as a [known] function of the [unknown] parameter $\theta$ and a fixed distribution [unobserved] random vector $\xi$, $$(X_1,\ldots,X_n)=\Psi(\xi,\theta)$$ This representation is eminently useful for simulation, either for the production of (pseudo-) samples from $F_\theta$ as in the inverse cdf approach, or for achieving "perfect" simulation. And for conducting inference by inverting the equation in $\theta$ into a distribution as in fiducial inference. Called by Efron Fisher's biggest blunder.

To relate with the previous answers, this [distributional] dependence of $X$ on [the true value of] $\theta$ does not imply that one can build a statistic that depends on $\theta$ because in the above equation both $\theta$ and $\xi$ are unobserved. Which is the whole point for conducting inference.

$\endgroup$
1
$\begingroup$

The confusion here stems from conflating a random variable with its distribution. To be clear about the issue, a random variable is not a function of the model parameters, but its distribution is.

Taking things back to their foundations, you have some probability space that consists of a sample space $\Omega$, a class of subsets on that space, and a class of probability measures $\mathbb{P}_\theta$ indexed by a model parameter $\theta$. Now, the random variable $X: \Omega \rightarrow \mathbb{R}$ is just a mapping defined on the domain $\Omega$. The random variable itself does not depend in any way on the parameter $\theta$, and so it is wrong to write it as a function $X(\theta)$. It is of course true that the probability distribution of $X$ depends on $\theta$, since the latter affects the probability measure over the sample space. However, it does not affect the sample space itself.$\dagger$

Consequently, when you are dealing with a statistic, which is just a function of the observed random variables, this also does not depend on $\theta$, but its distribution usually does. (If not, it is an ancillary statistic,)


$\dagger$ This treatment has taken $\theta$ as an index for the probability measure, but the same result occurs under a Bayesian treatment where $\theta$ is regarded as a random variable on $\Omega$ and the behaviour of $X$ is treated conditionally on the parameter.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.