14
$\begingroup$

This is the definition for statistic on wikipedia

More formally, statistical theory defines a statistic as a function of a sample where the function itself is independent of the sample's distribution; that is, the function can be stated before realization of the data. The term statistic is used both for the function and for the value of the function on a given sample.

I think I understand most of this definition, however the part - where the function is independent of the sample's distribution I haven't been able to sort out.

My understanding of statistic so far

A sample is a set of realizations of some number of independent, identically distributed (iid) random variables with distribution F (10 realizations of a roll of a 20-sided fair dice, 100 realizations of 5 rolls of a 6-sided fair dice, randomly draw 100 people from a population).

A function, whose domain is that set, and whose range is the real numbers (or maybe it can produce other things, like a vector or other mathematical object...) would be considered a statistic.

When I think of examples, mean, median, variance all make sense in this context. They are a function on set of realizations (blood pressure measurements from a random sample). I can also see how a linear regression model could be considered a statistic $y_{i} = \alpha + \beta \cdot x_{i}$ - is this not just a function on a set of realizations?

Where I'm confused

Assuming that my understanding from above is correct, I haven't been able to understand where a function might not be independent of the sample's distribution. I've been trying to think of an example to make sense of it, but no luck. Any insight would be much appreciated!

$\endgroup$
43
$\begingroup$

That definition is a somewhat awkward way to state it. A "statistic" is any function of the observable values. All that definition means is that a statistic is a function only of the observable values, not a function of the distribution or any of its parameters. For example, if $X_1, X_2, ..., X_n \sim \text{N}(\mu, 1)$ then a statistic would be any function $T(X_1,...,X_n)$ whereas a function $H(X_1,....,X_n, \mu)$ would not be a statistic, since it depends on $\mu$. Here are some further examples:

$$\begin{equation} \begin{aligned} \text{Statistic} & & & & & \bar{X}_n = \frac{1}{n} \sum_{i=1}^n X_i, \\[12pt] \text{Statistic} & & & & & S_n^2 = \frac{1}{n} \sum_{i=1}^n (X_i - \bar{X}_n)^2, \\[12pt] \text{Not a statistic} & & & & & D_n = \bar{X}_n - \mu, \\[12pt] \text{Not a statistic} & & & & & p_i = \text{N}(x_i | \mu, 1), \\[12pt] \text{Not a statistic} & & & & & Q = 10 \mu. \\[12pt] \end{aligned} \end{equation}$$

Every statistic is a function only of the observable values, and not of their distribution or its parameters. So there are no examples of a statistic that is a function of the distribution or its parameters (any such function would not be a statistic). However, it is important to note that the distribution of a statistic (as opposed to the statistic itself) will generally depend on the underlying distribution of the values. (This is true for all statistics other than ancillary statistics.)


What about a function where the parameters are known? In the comments below, Alecos asks an excellent follow-up question. What about a function that uses a fixed hypothesised value of the parameter? For example, what about the statistic $\sqrt{n} (\bar{x} - \mu)$ where $\mu = \mu_0$ is taken to be equal to a known hypothesised value $\mu_0 \in \mathbb{R}$. Here the function is indeed a statistic, so long as it is defined on the appropriately restricted domain. So the function $H_0: \mathbb{R}^n \rightarrow \mathbb{R}$ with $H_0(x_1,...,x_n) = \sqrt{n} (\bar{x} - \mu_0)$ would be a statistic, but the function $H: \mathbb{R}^{n+1} \rightarrow \mathbb{R}$ with $H(x_1,...,x_n, \mu) = \sqrt{n} (\bar{x} - \mu)$ would not be a statistic.

$\endgroup$
  • 1
    $\begingroup$ Very helpful answer, considering the underlying statistical parameter as part of the non-statistic was particularly helpful. $\endgroup$ – Jake Kirsch Mar 11 at 10:56
  • 4
    $\begingroup$ @CarlWitthoft I don't get your point. If it's a function of the observable values, then it's a statistic. It may be a function of a smaller subset of the values; that can still be a useful thing to consider. If you want to estimate the mean and you have $10^{10}$ observations, you might still look at $(X_1+X_2+\dots+X_{1000})/1000$ if the cost of processing data is high and the cost of error is small. Or for some reason you might want to consider two independent estimates of the mean, and could consider $(X_1+\dots+X_{n/2})/(n/2)$ and $(X_{n/2+1}+\dots+X_n)/(n/2)$. These are still statistics. $\endgroup$ – James Martin Mar 11 at 14:06
  • 4
    $\begingroup$ Those examples seem entirely valid to me. Are you saying the idea of dividing data into a training set and a validation set is not valid? $\endgroup$ – James Martin Mar 11 at 14:53
  • 2
    $\begingroup$ I'm a little confused by that as well. Let me attempt to describe @CarlWitthoft point. It would still be a statistic in terms of mathematical definition, but I could see a case where a consultant takes a 'statistic' of observations, but arbitrarily decides to remove a few results (consultants do this all the time right?). This would be 'valid' in the sense it's still a function on observations, however the way that statistic may be presented and interpreted likely wouldn't be valid. $\endgroup$ – Jake Kirsch Mar 11 at 15:41
  • 2
    $\begingroup$ @Carl Withhoft: With respect to the point you are making, it is important to distinguish between a statistic (which need not include all the data, and may not encompass all the information in the sample) and a sufficient statistic (which will encompass all the information with respect to some parameter). Statistical theory already has well-developed concepts like sufficiency that capture the idea that a statistic includes all relevant information in the sample. It is not necessary, or desirable, to try to build that requirement into the definition of a "statistic". $\endgroup$ – Ben Mar 11 at 21:11
4
$\begingroup$

I interpret that as saying that you should decide before you see the data what statistic you are going to calculate. So, for instance, if you're going to take out outliers, you should decide before you see the data what constitutes an "outlier". If you decide after you see the data, then your function is dependent on the data.

$\endgroup$
  • $\begingroup$ this is also helpful! So making a decision on which observations to include in the function after knowing what observations are available, which is more or less what I was describing in my comment on the previous answer. $\endgroup$ – Jake Kirsch Mar 11 at 19:37
  • 2
    $\begingroup$ (+1) It might be worth noting that this important because if you define a rule a prior about what constitutes a data point that will be dropped, it is (relatively) easy to derive a distribution for statistic (i.e., truncated mean, etc.). It's really hard to derive a distribution for a measure that involves dropping data points for reasons that are not cleanly defined before hand. $\endgroup$ – Cliff AB Mar 11 at 23:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.