# How to define a sample and a test statistic in Kolmogorov's theory?

Both a Random variable an a Statistic (understood as "single measure of some attribute of a sample") map to the real line the results of an outcome of a random experiment (in the case of a random variable), or a sample of such random experiments (in the case of a statistic).

In both instances, and despite the name (“random”), the functions are deterministic: Once the outcome of the actual random experiment is actualized, the mapping to the real line that defines a random variable as a function, $X: \Omega \rightarrow \mathbb R$, is clearly specified, either verbally (e.g. "number of heads in four coin tosses") or algebraically (e.g. $ln(\text{GDP})$). This seems to be equivalent to a statistic as a function applied to a sample (e.g. $\bar x= \frac{\sum x_i}{n}$), at least in its deterministic nature. Also, the interchangeability of both concepts seems reinforced by the this line in the Wikipedia entry: A statistic is an observable random variable.

A straightforward answer would be to point out the summary use of sample statistics, such as the mean or the variance, as well as the use of test statistics for inference.

So it may be at this point confusing why I am asking this question at all. It stems from a post yesterday on confidence intervals as random variables, and a response I ultimately decided to withdraw after this doubt became nagging (I will reinstate it just in case someone is interested in understanding the possible confusion better).

Question: Are these two concepts interrelated enough to use them interchangeable in explaining concepts such as confidence intervals? Is it OK to consider their main difference their reference to a single outcome (random variable) versus a sample (statistic)?

• Functions of random variables are also random variables.
– Sycorax
Aug 3, 2016 at 1:22
• @General Abrial Thank you. And I suppose the leap from single outcome to sample doesn't violate any probability tenant to invalidate the statement on my previous answer (linked at the end of OP)... do you agree? Aug 3, 2016 at 1:24
• I think your answer is technically accurate, but I do find it hard to follow and to relate back to the original question. But I'm also very tired. I'll take a look at it tomorrow morning.
– Sycorax
Aug 3, 2016 at 1:32
• It seems to me the Wikipedia characterization, by clearly indicating how a statistic is a special case of a random variable, does the opposite of reinforcing the "interchangeability of both concepts." In a HMM, for instance, the unobservable random variables in the model are by definition not statistics.
– whuber
Aug 3, 2016 at 13:34

A probability space is defined as a tripple $(\Omega, \mathcal{F},P)$, where $\Omega$ is the set of possible outcomes, $\mathcal{F}$ a $\sigma$-algebra on $\Omega$ and $P$ a probability measure on $\mathcal{F}$.

As you say, a random variable $X$ is a map from $\Omega$ to $\mathbb{R}$ such that for any Borel set $B$ in $\mathbb{R}$ it holds that $X^{-1}(B) \in \mathcal{F}$. Because of the latter propery, it holds that, for any Borel set $B$, there exists an event $E \in \mathcal{F}$ such that $X(E)=B$. Therefore we can measure any Borel set with a measure (depending on $X$) $\mu_X$ by defining $\mu_X(B)=P(E)$ where $E$ is the event supra. In other words, to ''measure'' B, look for its inverse image under $X^{-1}$, which (by definition of $X$) belongs to $\mathcal{F}$, and, as this inverse $E=X^{-1}(B)$ is in $\mathcal{F}$, we can measure it with the probability measure $P$, i.e. $\mu_X(B)=P(E)=P(X^{-1}(B))$

If $\mathcal{B}$ is the set of all Borel sets, then it can be shown that $(\mathbb{R}, \mathcal{B},\mu_X)$ is also a probability space. The map $X$ is called a random variable, and the function $\mu_X$ is called the distribution of $X$.

This formally defines a random variable and its distribution. Examples are a normal random variable, Binomial random variables, etc. (see this link for detail on the Binomial random variable).

Let us take the normal random variable, with mean $\mu$ and standard deviation $\sigma$ as an example i.e. $X \sim N(\mu, \sigma$). A (random) sample of size $n$ are just $n$ random outcomes $x_1, x_2, \dots, x_n$ from the distribution of $X$.

These outcomes are random, so if we redo the random draws, we will find ''other'' values $y_1, y_2, \dots, y_n$. Therefore the sample average $\bar{x}=\frac{1}{n}\sum_{i=1}^n x_i$ is also ''random''.

This sample average is a so-called ''test statistic'' and it is a special case of a random variable, namely a random variable that is derived from the normal random variable $X$.

Let's see how these two are defined formally, and let's , for simplicity, say that the Borel sets are all the intervals in $\mathbb{R}$. Then the normal random variable $X$ has $\Omega=\mathbb{R}$, $\mathcal{F}$ is the set of all intervals (I simplified here, it should be a $\sigma$-algebra), and the measure $P$ for an interval $[a,b]$ is $P([a,b]=\int_a^b f(x) dx$ where $f$ is the density of a normal variable with mean $\mu$ and standard deviation $\sigma$.

Your test statistic $\bar{X}$ is another random variable, derived from $X$, with the same $\Omega$, the same $\sigma$-algebra, but with another measure $P'$ where $P'([a,b]=\int_a^b f'(x) dx$ and $f'$ is the density of a normal variable with mean $\mu$ and standard deviation $\frac{\sigma}{\sqrt{n}}$.

So, to answer your question, a test statistic is a special case of a random variable. The test statistic is thus a random variable, related to the random variable from which the sample (used to compute the test statistic) was drawn.

I would not go that far as saying that a sample relates to a test statistic like an outcome relates to a random variable. A test statistic is a random variable (cfr supra) however in my opinion the sample ($x_1, x_2, \dots , x_n$) is not the outcome of a test statistic ($\bar{X}$). I would say that the sample average is an outcome of the random variable $\bar{X}$.

The sample itself is an outcome of another random variable with outcomes in the product probability space with outcome set $\Omega \times \Omega \times \dots \times \Omega$ (and ''induced'' sigma-algebra and probability measures).

• Really well-written, and enlightening. I think the real answer is in the last three paragraphs, but the preceding review is really good. Thank you! Aug 3, 2016 at 12:25
• The line "Because of the latter propery, it holds that, for any Borel set $B$, there exists an event $E\in\mathcal{F}$ such that $X(E)=B$" is wrong!!! For example, constant map $X:\Omega\rightarrow \mathbb{R}$, $X(\omega)=0$ is a random variable. Let $B=\{1\}$, which is a Borel set. However, there is no $E\in\mathcal{F}$ such that $X(E)=B$. Jun 25, 2021 at 19:16