# What is the difference between an estimator and a statistic?

I learned that a statistic is an attribute you can obtain from samples.Taking many samples of same size, calculating this attribute for all of them and plotting the pdf, we get the distribution of the corresponding attribute or the distribution of the corresponding statistics.

I also heard that statistics are made to be estimators, how do these two concepts differ?

• Thanks for all the answears ... The concept is a lot more clear to me now.. Jan 15, 2013 at 16:19
• There are some rather narrow definitions in this thread. I have no difficulty with treating any plot (e.g. a scatter plot or a quantile plot) based on a sample as a sample statistic, for example, and thinking in terms of a sampling distribution of such plots. The idea of a line-up of such plots -- recently reinvented by several but going back at least to Shewhart -- is one manifestation. A statistic in the sense of a single number calculated from a sample is no doubt the case first met by learners,, the simplest case and the most important case, but there is no need to make that a definition. Aug 16, 2021 at 8:37

## Definition

From Wikipedia:

A statistic [...] is a single measure of some attribute of a sample (e.g., its arithmetic mean value).

And

[A]n estimator is a rule for calculating an estimate of a given quantity [of the underlying distribution] based on observed data.

The important difference is:

• A statistic is a function of a sample.
• An estimator is a function of a sample related to some quantity of the distribution.

For what "Quantity" means, see section below. It's simply a function of the distribution.

## A statistic is not an estimator

An estimator is a statistic with something added. To turn a statistic into an estimator, you simply spell out which target quantity you want to estimate. This is confusing, because you do not add anything "real" to the statistic, but only some intend.

To see that the difference is important, you have to realize that you cannot calculate the properties of an estimator (e.g. bias, variance, etc.) for a mere statistic. To calculate bias, you have to find the difference between the value your statistic gives you and the true value. Only an estimator comes with a "true value" which allows to compute a bias. A statistic is merely a function of the data, and it is neither right nor wrong.

### Different estimators based on the same statistic

You can spell out different target quantities for the same statistic, resulting in different estimators. Each such estimator has its own bias, although they all are (based on) the same value, the same statistic.

• You can use sample mean as an estimator for distribution mean. This estimator has zero bias.
• You can also use sample mean as an estimator for distribution variance. This estimator is biased for most distributions.

So saying "sample mean is unbiased" does not make sense. Sample mean is unbiased when you use it to estimate distribution mean. But at the same time it is biased when using it to estimate distribution variance.

## Quantities of distributions and quantities of samples

A quantity is a function of the distribution. If you only have a single distribution, and no class, then the quantity is a single value (the domain of the function has one element).

Here quantity refers to some property of the distribution, which is usually unknown and thus has to be estimated. This is in contrast to a statistic, which is a property of a sample, e.g. the distribution mean is a quantity of your distribution, while the sample mean is a statistic (a quantity of your sample).

• There's nothing overtly wrong with these quotations, but they leave me puzzling over what exactly is meant by "quantity." For instance, the quotations do not appear to rule out the possibility that a "quantity" is another statistic based on the same data or perhaps is another statistic based on a separate set of similar data. (In the latter case the first statistic might be used as a predictor. In the former case I don't think there's a name for it, but it definitely is not "estimator.")
– whuber
Jan 15, 2013 at 13:27
• @whuber See edit. Initially I wanted to give a short answer... :( Jan 15, 2013 at 14:09
• Presumably the sample mean and sample median will only estimate the same underlying value if the distribution is one where median=mean... Jan 15, 2013 at 16:26
• My criticism makes less sense in light of your edit. I was merely saying that in many distributions median != mean, so the sample median and sample mean will not converge to the same value in such cases (i.e., don't estimate the same thing). Jan 17, 2013 at 0:59
• @Stumpy I think you have a slight misconception here. It does not matter whether the median and mean "converge" to the same thing (or to anything at all). To clarify this, let me be a little ridiculous: I may, if I wish, use the sample variance to estimate the mean. There is absolutely no theoretical restriction--nor can there be--that says I cannot do this. My procedure fulfills all parts of the definition: the sample variance is truly a statistic and the mean is truly a property of the underlying distribution. For the definitions, it is irrelevant that this is (often) a terrible procedure.
– whuber
Jan 17, 2013 at 20:01

This thread is a little old, but it appears that Wikipedia may have changed its definition and if it's accurate, it explains it more clearly for me:

An "estimator" or "point estimate" is a statistic (that is, a function of the data) that is used to infer the value of an unknown parameter in a statistical model.

So a statistic refers to the data itself and a calculation with that data. While an estimator refers to a parameter in a model.

If I understand it correctly, then, the mean is a statistic and may also be an estimator. The mean of a sample is a statistic (sum of the sample divided by the sample size). The mean of a sample is also an estimator of the mean of the population, assuming it's normally distributed.

I'd ask @whuber and others who really know this stuff if the (new?) Wikipedia quote is accurate.

• +1 I think you have it basically right. You might be interested to know that the target of an estimator does not necessarily have to be a particular "parameter" of a model: it can be any property of the model, such as a function of its parameters. For instance, $\mu^2$ is not a parameter for a Normal$(\mu,\sigma^2)$ model, but it can be estimated.
– whuber
Jun 15, 2013 at 16:47

"6" is an example of an estimator. Say your question was, "what is the slope of the best linear function mapping x to y?" Your answer could be "6". Or it could be $(X'X)^{-1}X'Y$. Both are estimators. Which one is better is left to you to decide.

A really good TA once explained the concept of an estimator to me that way.

Basically, an estimator is a thing that you apply to data to get a quantity that you don't know the value of. You know the value of a statistic -- it is a function of the data with no "best" or "optimal" about it. There is no "best" mean. There is just a mean.

Say you have a dataset on number of goats owned per person, and each person's happiness. You're interested in how people's happiness changes with the number of goats they own. An estimator can help you to estimate that relationship from your data. Statistics are just functions of the data that you have. For example, the variance of goat ownership may equal 7. Te forula for calculating variance would be identical between goats and toasters, or whether you're interested in happiness or propensity to get cancer. In that sense, all sensible estimators are statistics.

Since other answers saying that they are the same give no authoritative reference, let me give you two quotes from Statistical inference handbook by Casella and Berger:

Definition 5.2.1 Let $X_1,\dots,X_n$ be a random sample of size $n$ from a population and let $T(x_1,\dots,x_n)$ be a real-valued or vector-valued function whose domain includes the sample space of $(X_1,\dots,X_n)$. Then the random variable or random vector $Y = T(X_1,\dots,X_n)$ is called statistic. The probability distribution of statistic $Y$ is called sampling distribution of $Y$.

and

Definition 7.1.1 A point estimator is any function $W(X_1,\dots,X_n)$ of a sample; that is, any statistic is a point estimator.

I am not saying in here that this is the definite answer to the question, since I seem to agree with the two most upvoted answers that suggest that there is a difference, just giving a reference that says the opposite to highlight that this is not a clear-cut case.

Interesting question. Estimators and statistics do not need to be different things, though. They are different concepts.

A statistic is a function (in broad terms) in which the input is (statistical) data. The effect is that you gain a result, usually a number, from this statistic. In a more abstract term, a statistic may yield more than one number. The statistic depends on the data, but the procedure is deterministic. So the statistic may be: "Sum all numbers and divide by the count" or, in the broader sense "take the gdp data and prepare a report on it".
In the statistical sense we are of course talking about a mathematical function as a statistic.

The significance of this is that if you know properties of the data you input (for example it beeing a random variable), then you can calculate the properties of your statistic, without actually putting in empirical data.

Estimators are estimators because of you intent: to estimate a property. As it turns out, some statistics are good estimators.
For example if you pull data points out of a pool of i.i.d. variables, then the arithmetic mean - a statistic based on the data you pull, will probably be a good estimator for the expected value of that distribution. But then again any thing that produces an estimate is an estimator.

In practice, estimators that you use will be statistics, but there are statistics that aren't estimators. For example test-statistics - though one can argue about the semantics of this statement and to make matters worse, a test statistic may not only be but also include estimators. Though conceptually this doesn't have to be the case.

And of course you can have estimators that aren't statistics, though they are probably not very good at estimating.

• Could you elaborate a little on that last sentence? For instance, consider an iid sample of size $2n$. I will estimate the median of the population by using a coin flip to choose among the $n$th and $n+1$st largest values in the sample. According to your definition this is not a statistic, because it is not a "deterministic" procedure (although it is a statistic according to a common more general definition). It also is a reasonably good estimator. So I am wondering just what kind of object you have in mind when you refer to an "estimator" that is not a "statistic."
– whuber
Jan 15, 2013 at 13:31
• Yeah I would argue that "choosing a value" is the deterministic statistic and everything beforehand is related to modification of the sample you chose from. Then again since the "procedure" if you will - is deterministic I may just allow stochastic elements such this in my definition of statistic... Point beeing that estimators which are not a statistic could be at least those which are independent of any data. For example the number "6" in the answer below. Please note that I did not say that non-statistic estimators are necessarily bad.
– IMA
Jan 17, 2013 at 7:52
• I think perhaps you are making too many fine distinctions which are unnecessary and, in the end, complicate your exposition. For example, "1/2" is a great estimator of the parameter of a Bernoulli variable (it is minimax for quadratic loss), so it would be a shame to rule it out just because it is independent of the data. (That would be analogous to ruling out squares as examples of rectangles in Euclidean geometry: you could do that, but that would then double the lengths of most statements concerning properties of rectangles.) It similarly helps not to rule out randomized statistics.
– whuber
Jan 17, 2013 at 15:28
• I don't think we are really talking about the same thing. Where do I rule out anything? If one-half is a great estimator, then it is a case where it is. I just don't think that the majority of possible estimators not beeing statistics are pretty great. For a Bernoulli variable "1/2" is good. But -quite- a few other estimators from the class "A real number" are not very good, wouldn't you agree? On the matter of randomized statistics still based on data- I didn't rule it out as I would still say you will require a deterministic procedure. But I concede that I should add this above.
– IMA
Jan 18, 2013 at 8:08

I think a better understanding about what is a sample helps.

[Updated: Sample is a very broad concept, I was talking about "the random sample" . I don't know whether an estimator makes sense or not when the sample is not random.]

from wikipedia:

A random sample is defined as a sample where each individual member of the population has a known, non-zero chance of being selected as part of the sample.

An estimator is a function of a sample. A sample is actually a set of (say, $n$) i.i.d. random variables. That means an estimator is also a function of random variables. An estimator defines a measurement, but not the values of an actual measurement. But we can call it, "the rule for estimating a given quantity based on observed data." Because based on $n$ specific experiments, we can have $n$ specific values for the $n$ i.i.d. random variables. And we get a specific value of the size-$n$ sample.

We replace the sample in the estimator by the value of the sample. We get a value of the estimator, this is a specific measure. And this specific measure is a statistic.

(Check this link for the definition of an estimator, the last sentence reveals why we are always confused.)

The Goal of This Piece of Writing:

What I want to do here is to provide you with the similarities and differences between the two intimately related concepts called "statistic" and "estimator". However, I do not want to go through the differences between a parameter and a statistic, which I assume is clear enough to everyone who is struggling with the differences between a statistic and an estimator. If it is not the case for you, you need to study earlier posts first, and then start studying this post.

Relationship:

Basically, any real-valued function of observable random variables in a sample is called a statistic. There are some statistics that if they are well designed, and have some good properties (e.g. consistency, ... ), they can be used to estimate the parameters of the underlying distribution of the population. Therefore, statistics are a large set, and estimators are a subset inside the set of statistics. Hence, every estimator is a statistic, but not every statistic is an estimator.

Similarities:

Speaking of the similarities, as mentioned earlier, both are functions of random variables. In addition, both have distributions called "sampling distributions."

Differences:

Speaking of the differences, they are different in terms of their goals and tasks. The goals and tasks of a statistic could be summarizing the information in a sample (by using sufficient statistics), and sometimes doing hypothesis test, etc. In contrast, the primary goal and task of an estimator, as its name implies, is to estimate the parameters of the population being studied. It is important to mention that there are a wide variety of estimators, each of which has its own computational logic behind, such as MOMEs, MLEs, OLS estimators and so on. Another difference between these two concepts has to do with their desired properties. While one of the most desired properties of a statistic is "sufficiency", the desired properties of an estimator are things like "consistency", "unbiasedness", "precision", etc.

Caution:

Therefore, you need to be careful about using terminology correctly when dealing with statistics and estimators. For instance, it does not make much sense to talk about the biasedness of a mere statistic, which is by no means an estimator, because there is no parameter involved in such a context in order for us to be able to calculate the bias, and talk about it. Thus, you need to be careful about the terminology!

The Bottom Line:

To sum up, any function of observable random variables in a sample is a statistic. If a statistic has capability to estimate a parameter of a population, then we call it an estimator (of the parameter of interest). However, there are some statistics that are not designed to estimate parameters, so these statistics are not estimators, and here we call them "mere statistics".

What I offered above is the way I look at and think of these two concepts, and I tried my best to put it in simple words. I hope it helps!

New answer to an old Q:

Definition 1. A statistic is a function that maps each sample to a real number.

Every estimator is a statistic.

But we tend to call only those statistics that are used to generate estimates ("guesses") some parameter an estimator.

So for example, the t-statistic and the sample mean are BOTH statistics. The sample mean is also an estimator (because we often use it to estimate the true population mean).

In contrast, we rarely/never call the t-statistic an estimator, because we rarely/never use it to estimate any parameter.

In the example below, $$P$$ is a statistic, but not an estimator. While $$Q$$ is both a statistic and an estimator.



$$\underline{\text{Example}}$$

Suppose our parameter-of-interest is the average outcome $$\theta$$ of a die-roll.

$$\theta$$ is some fixed real number that is perhaps known only to God. Nonetheless, we can try to estimate it.

Here's one possible method. We roll a die 3 times.

A sample is any $$\textbf{s}=\left(x_1,x_2,x_3\right)$$, where $$x_1$$ is the outcome of the first roll, $$x_2$$ that of the second, and $$x_3$$ that of the third.

Here are three examples of samples: $$\textbf{s}_1=\left(5,4,1\right)$$, $$\textbf{s}_2=\left(4,1,6\right)$$, and $$\textbf{s}_3=\left(6,3,2\right)$$.

Here are two examples of statistics $$P$$ and $$Q$$ (remember that a statistic is simply a function). Define $$P$$ and $$Q$$ by: For any $$\textbf{s}=\left(x_1,x_2,x_3\right)$$, $$P(\textbf{s})=\frac{x_1}{\ln(x_2+x_3)},$$ $$Q(\textbf{s})=\frac{x_1+x_2+x_3}{3}.$$

The statistic $$P$$ is a rather-bizarre statistic and is probably not very useful for anything. Nonetheless, it is a statistic all the same, simply because it satisfies the definition of a statistic (it is a function that maps each sample to a real number).

$$Q$$ is also a statistic. But in addition, it is also an estimator for the parameter $$\theta$$.

(We could, of course, claim that $$P$$ is also an estimator for $$\theta$$. But it would be a very poor estimator that no one would want to use.)

• This answer is headed in a good direction. "Definition 2," though, does not appear to be a valid definition, because of its circularity (it defines "estimator" in terms of "estimate" without explaining the latter). For it to be effective you need to explain what an "estimate of a parameter" is in sufficient detail and clarity that people can formulate quantitative measurements of how well an estimator works.
– whuber
Jun 21, 2016 at 13:27
• @whuber: I'm trying to keep it simple. A parameter is any real number (e.g. the average outcome $\theta$ of a die roll). Informally, an estimate for a parameter is simply a "guess" of what a parameter is. An estimate is thus simply also a real number. (E.g., an estimate of $\theta$ is $5$.) // The question of "how to formulate quantitative measurements of how well an estimator works" is entirely distinct from the simpler and more basic question of the distinction between a statistic and an estimator. Which is the question here.
– user46481
Jun 22, 2016 at 12:16
• Unfortunately, as I was trying to suggest, something essential seems to have been lost in the simplification, because your second definition does not distinguish an estimator from any other statistic at all.
– whuber
Jun 22, 2016 at 13:32
• @whuber: That's right. Formally, an estimator is simply a statistic. But we tend to use the word "estimator" to refer to a statistic if that statistic is used to estimate some parameter-of-interest. I have edited my answer to clarify this point.
– user46481
Jun 23, 2016 at 2:25

In hypothesis testing :

A test-statistic is about hypothesis testing. A test-statistic is a random variable given/under the null hypothesis. Now, some may call a statistic the value/measure of the test-statistic given the sample.

With these two you can get the p-value which is a measure that helps to reject or not reject the null hypothesis. All in all, a statistic is an estimation of how far/close to your hypothesis.