I learned that a statistic is an attribute you can obtain from samples.Taking many samples of same size, calculating this attribute for all of them and plotting the pdf, we get the distribution of the corresponding attribute or the distribution of the corresponding statistics.
I also heard that statistics are made to be estimators, how do these two concepts differ?
From Wikipedia:
A statistic [...] is a single measure of some attribute of a sample (e.g., its arithmetic mean value).
And
[A]n estimator is a rule for calculating an estimate of a given quantity [of the underlying distribution] based on observed data.
The important difference is:
For what "Quantity" means, see section below. It's simply a function of the distribution.
An estimator is a statistic with something added. To turn a statistic into an estimator, you simply spell out which target quantity you want to estimate. This is confusing, because you do not add anything "real" to the statistic, but only some intend.
To see that the difference is important, you have to realize that you cannot calculate the properties of an estimator (e.g. bias, variance, etc.) for a mere statistic. To calculate bias, you have to find the difference between the value your statistic gives you and the true value. Only an estimator comes with a "true value" which allows to compute a bias. A statistic is merely a function of the data, and it is neither right nor wrong.
You can spell out different target quantities for the same statistic, resulting in different estimators. Each such estimator has its own bias, although they all are (based on) the same value, the same statistic.
So saying "sample mean is unbiased" does not make sense. Sample mean is unbiased when you use it to estimate distribution mean. But at the same time it is biased when using it to estimate distribution variance.
A quantity is a function of the distribution. If you only have a single distribution, and no class, then the quantity is a single value (the domain of the function has one element).
Here quantity refers to some property of the distribution, which is usually unknown and thus has to be estimated. This is in contrast to a statistic, which is a property of a sample, e.g. the distribution mean is a quantity of your distribution, while the sample mean is a statistic (a quantity of your sample).
This thread is a little old, but it appears that Wikipedia may have changed its definition and if it's accurate, it explains it more clearly for me:
An "estimator" or "point estimate" is a statistic (that is, a function of the data) that is used to infer the value of an unknown parameter in a statistical model.
So a statistic refers to the data itself and a calculation with that data. While an estimator refers to a parameter in a model.
If I understand it correctly, then, the mean is a statistic and may also be an estimator. The mean of a sample is a statistic (sum of the sample divided by the sample size). The mean of a sample is also an estimator of the mean of the population, assuming it's normally distributed.
I'd ask @whuber and others who really know this stuff if the (new?) Wikipedia quote is accurate.
"6" is an example of an estimator. Say your question was, "what is the slope of the best linear function mapping x to y?" Your answer could be "6". Or it could be $(X'X)^{-1}X'Y$. Both are estimators. Which one is better is left to you to decide.
A really good TA once explained the concept of an estimator to me that way.
Basically, an estimator is a thing that you apply to data to get a quantity that you don't know the value of. You know the value of a statistic -- it is a function of the data with no "best" or "optimal" about it. There is no "best" mean. There is just a mean.
Say you have a dataset on number of goats owned per person, and each person's happiness. You're interested in how people's happiness changes with the number of goats they own. An estimator can help you to estimate that relationship from your data. Statistics are just functions of the data that you have. For example, the variance of goat ownership may equal 7. Te forula for calculating variance would be identical between goats and toasters, or whether you're interested in happiness or propensity to get cancer. In that sense, all sensible estimators are statistics.
Since other answers saying that they are the same give no authoritative reference, let me give you two quotes from Statistical inference handbook by Casella and Berger:
Definition 5.2.1 Let $X_1,\dots,X_n$ be a random sample of size $n$ from a population and let $T(x_1,\dots,x_n)$ be a real-valued or vector-valued function whose domain includes the sample space of $(X_1,\dots,X_n)$. Then the random variable or random vector $Y = T(X_1,\dots,X_n)$ is called statistic. The probability distribution of statistic $Y$ is called sampling distribution of $Y$.
and
Definition 7.1.1 A point estimator is any function $W(X_1,\dots,X_n)$ of a sample; that is, any statistic is a point estimator.
I am not saying in here that this is the definite answer to the question, since I seem to agree with the two most upvoted answers that suggest that there is a difference, just giving a reference that says the opposite to highlight that this is not a clear-cut case.
Interesting question. Estimators and statistics do not need to be different things, though. They are different concepts.
A statistic is a function (in broad terms) in which the input is (statistical) data. The effect is that you gain a result, usually a number, from this statistic. In a more abstract term, a statistic may yield more than one number.
The statistic depends on the data, but the procedure is deterministic. So the statistic may be: "Sum all numbers and divide by the count" or, in the broader sense "take the gdp data and prepare a report on it".
In the statistical sense we are of course talking about a mathematical function as a statistic.
The significance of this is that if you know properties of the data you input (for example it beeing a random variable), then you can calculate the properties of your statistic, without actually putting in empirical data.
Estimators are estimators because of you intent: to estimate a property.
As it turns out, some statistics are good estimators.
For example if you pull data points out of a pool of i.i.d. variables, then the arithmetic mean - a statistic based on the data you pull, will probably be a good estimator for the expected value of that distribution. But then again any thing that produces an estimate is an estimator.
In practice, estimators that you use will be statistics, but there are statistics that aren't estimators. For example test-statistics - though one can argue about the semantics of this statement and to make matters worse, a test statistic may not only be but also include estimators. Though conceptually this doesn't have to be the case.
And of course you can have estimators that aren't statistics, though they are probably not very good at estimating.
I think a better understanding about what is a sample helps.
[Updated: Sample is a very broad concept, I was talking about "the random sample" . I don't know whether an estimator makes sense or not when the sample is not random.]
from wikipedia:
A random sample is defined as a sample where each individual member of the population has a known, non-zero chance of being selected as part of the sample.
An estimator is a function of a sample. A sample is actually a set of (say, $n$) i.i.d. random variables. That means an estimator is also a function of random variables. An estimator defines a measurement, but not the values of an actual measurement. But we can call it, "the rule for estimating a given quantity based on observed data." Because based on $n$ specific experiments, we can have $n$ specific values for the $n$ i.i.d. random variables. And we get a specific value of the size-$n$ sample.
We replace the sample in the estimator by the value of the sample. We get a value of the estimator, this is a specific measure. And this specific measure is a statistic.
(Check this link for the definition of an estimator, the last sentence reveals why we are always confused.)
The Goal of This Piece of Writing:
What I want to do here is to provide you with the similarities and differences between the two intimately related concepts called "statistic" and "estimator". However, I do not want to go through the differences between a parameter and a statistic, which I assume is clear enough to everyone who is struggling with the differences between a statistic and an estimator. If it is not the case for you, you need to study earlier posts first, and then start studying this post.
Relationship:
Basically, any real-valued function of observable random variables in a sample is called a statistic. There are some statistics that if they are well designed, and have some good properties (e.g. consistency, ... ), they can be used to estimate the parameters of the underlying distribution of the population. Therefore, statistics are a large set, and estimators are a subset inside the set of statistics. Hence, every estimator is a statistic, but not every statistic is an estimator.
Similarities:
Speaking of the similarities, as mentioned earlier, both are functions of random variables. In addition, both have distributions called "sampling distributions."
Differences:
Speaking of the differences, they are different in terms of their goals and tasks. The goals and tasks of a statistic could be summarizing the information in a sample (by using sufficient statistics), and sometimes doing hypothesis test, etc. In contrast, the primary goal and task of an estimator, as its name implies, is to estimate the parameters of the population being studied. It is important to mention that there are a wide variety of estimators, each of which has its own computational logic behind, such as MOMEs, MLEs, OLS estimators and so on. Another difference between these two concepts has to do with their desired properties. While one of the most desired properties of a statistic is "sufficiency", the desired properties of an estimator are things like "consistency", "unbiasedness", "precision", etc.
Caution:
Therefore, you need to be careful about using terminology correctly when dealing with statistics and estimators. For instance, it does not make much sense to talk about the biasedness of a mere statistic, which is by no means an estimator, because there is no parameter involved in such a context in order for us to be able to calculate the bias, and talk about it. Thus, you need to be careful about the terminology!
The Bottom Line:
To sum up, any function of observable random variables in a sample is a statistic. If a statistic has capability to estimate a parameter of a population, then we call it an estimator (of the parameter of interest). However, there are some statistics that are not designed to estimate parameters, so these statistics are not estimators, and here we call them "mere statistics".
What I offered above is the way I look at and think of these two concepts, and I tried my best to put it in simple words. I hope it helps!
New answer to an old Q:
Definition 1. A statistic is a function that maps each sample to a real number.
Every estimator is a statistic.
But we tend to call only those statistics that are used to generate estimates ("guesses") some parameter an estimator.
So for example, the t-statistic and the sample mean are BOTH statistics. The sample mean is also an estimator (because we often use it to estimate the true population mean).
In contrast, we rarely/never call the t-statistic an estimator, because we rarely/never use it to estimate any parameter.
In the example below, $P$ is a statistic, but not an estimator. While $Q$ is both a statistic and an estimator.
$$$$
$$\underline{\text{Example}}$$
Suppose our parameter-of-interest is the average outcome $\theta$ of a die-roll.
$\theta$ is some fixed real number that is perhaps known only to God. Nonetheless, we can try to estimate it.
Here's one possible method. We roll a die 3 times.
A sample is any $\textbf{s}=\left(x_1,x_2,x_3\right)$, where $x_1$ is the outcome of the first roll, $x_2$ that of the second, and $x_3$ that of the third.
Here are three examples of samples: $\textbf{s}_1=\left(5,4,1\right)$, $\textbf{s}_2=\left(4,1,6\right)$, and $\textbf{s}_3=\left(6,3,2\right)$.
Here are two examples of statistics $P$ and $Q$ (remember that a statistic is simply a function). Define $P$ and $Q$ by: For any $\textbf{s}=\left(x_1,x_2,x_3\right)$, $$P(\textbf{s})=\frac{x_1}{\ln(x_2+x_3)},$$ $$Q(\textbf{s})=\frac{x_1+x_2+x_3}{3}.$$
The statistic $P$ is a rather-bizarre statistic and is probably not very useful for anything. Nonetheless, it is a statistic all the same, simply because it satisfies the definition of a statistic (it is a function that maps each sample to a real number).
$Q$ is also a statistic. But in addition, it is also an estimator for the parameter $\theta$.
(We could, of course, claim that $P$ is also an estimator for $\theta$. But it would be a very poor estimator that no one would want to use.)
In hypothesis testing :
A test-statistic is about hypothesis testing. A test-statistic is a random variable given/under the null hypothesis. Now, some may call a statistic the value/measure of the test-statistic given the sample.
With these two you can get the p-value which is a measure that helps to reject or not reject the null hypothesis. All in all, a statistic is an estimation of how far/close to your hypothesis.
This link may be useful.
