Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 8 down vote favorite
2

I learned that a statistic is an attribute you can obtain from samples.Taking many samples of same size, calculating this attribute for all of them and plotting the pdf, we get the distribution of the corresponding attribute or the distribution of the corresponding statistics.

I also heard that statistics are made to be estimators, how do these two concepts differ?

share|improve this question
Compare the first sentence of Estimator vs the first two sentences of Statistic. – Glen_b Jan 15 at 14:00
Thanks for all the answears ... The concept is a lot more clear to me now.. – gutto Jan 15 at 16:19

4 Answers

up vote 4 down vote

From Wikipedia:

A statistic [...] is a single measure of some attribute of a sample (e.g., its arithmetic mean value).

And

[A]n estimator is a rule for calculating an estimate of a given quantity [of the underlying distribution] based on observed data.

The important difference is:

  • A statistic relates to a sample.
  • An estimator relates to a sample with respect to some quantity.

(For what "Quantity" means, see section below.)

So you can turn a statistic into an estimator by simply spelling out the quantity you want it to estimate; the value doesn't change.

A statistic is not an estimator

The difference is very subtle and I think it is very common to call a statistic an estimator, e.g. saying sample mean is an estimator for distribution mean. This may be very confusing when you don't properly know both concepts. In particular you cannot calculate the properties of an estimator (e.g. bias) for a mere statistic. Saying "(uncorrected) sample variance is biased" doesn't make much sense, because it is only biased in its function of an estimator for the distribution variance. Thus, formally a statistic cannot be an estimator because you cannot do everything with a statistic which you can do with an estimator (e.g. compute its bias).

Also you can see, that the same statistic can be multiple estimators. E.g. you can estimate both the mean and the median of the distribution by, e.g. the sample mean (whether this is a good idea is up to you). Both 'estimators' will have different properties (e.g. different error, different bias), still underyling them is the same statistic, the same value.

Quantities of distributions and quantities of samples

Here quantity refers to some property of the distribution, which is usually unknown and thus has to be estimated. This is in contrast to a statistic, which is a property of a sample, e.g. the distribution mean is a quantity of your distribution, while the sample mean is a statistic (a quantity of your sample).

share|improve this answer
There's nothing overtly wrong with these quotations, but they leave me puzzling over what exactly is meant by "quantity." For instance, the quotations do not appear to rule out the possibility that a "quantity" is another statistic based on the same data or perhaps is another statistic based on a separate set of similar data. (In the latter case the first statistic might be used as a predictor. In the former case I don't think there's a name for it, but it definitely is not "estimator.") – whuber♦ Jan 15 at 13:27
@whuber See edit. Initially I wanted to give a short answer... :( – ziggystar Jan 15 at 14:09
+1 Your final paragraph says it all. – whuber♦ Jan 15 at 14:23
Presumably the sample mean and sample median will only estimate the same underlying value if the distribution is one where median=mean... – Stumpy Joe Pete Jan 15 at 16:26
1  
@StumpyJoePete Sample mean and sample median are statistics and as such don't estimate something unless you spell out what "something" is. I think you're thinking about "good" estimators. And indeed sample mean might be a rather bad estimator of the distribution mean for asymmetric distributions. But it can be treated as such an estimator in any case and you can compute its error and other properties of estimators. – ziggystar Jan 16 at 10:22
show 3 more comments
up vote 2 down vote

Interesting question. Estimators and statistics do not need to be different things, though. They are different concepts.

A statistic is a function (in broad terms) in which the input is (statistical) data. The effect is that you gain a result, usually a number, from this statistic. In a more abstract term, a statistic may yield more than one number. The statistic depends on the data, but the procedure is deterministic. So the statistic may be: "Sum all numbers and divide by the count" or, in the broader sense "take the gdp data and prepare a report on it".
In the statistical sense we are of course talking about a mathematical function as a statistic.

The significance of this is that if you know properties of the data you input (for example it beeing a random variable), then you can calculate the properties of your statistic, without actually putting in empirical data.

Estimators are estimators because of you intent: to estimate a property. As it turns out, some statistics are good estimators.
For example if you pull data points out of a pool of i.i.d. variables, then the arithmetic mean - a statistic based on the data you pull, will probably be a good estimator for the expected value of that distribution. But then again any thing that produces an estimate is an estimator.

In practice, estimators that you use will be statistics, but there are statistics that aren't estimators. For example test-statistics - though one can argue about the semantics of this statement and to make matters worse, a test statistic may not only be but also include estimators. Though conceptually this doesn't have to be the case.

And of course you can have estimators that aren't statistics, though they are probably not very good at estimating.

share|improve this answer
Could you elaborate a little on that last sentence? For instance, consider an iid sample of size $2n$. I will estimate the median of the population by using a coin flip to choose among the $n$th and $n+1$st largest values in the sample. According to your definition this is not a statistic, because it is not a "deterministic" procedure (although it is a statistic according to a common more general definition). It also is a reasonably good estimator. So I am wondering just what kind of object you have in mind when you refer to an "estimator" that is not a "statistic." – whuber♦ Jan 15 at 13:31
Yeah I would argue that "choosing a value" is the deterministic statistic and everything beforehand is related to modification of the sample you chose from. Then again since the "procedure" if you will - is deterministic I may just allow stochastic elements such this in my definition of statistic... Point beeing that estimators which are not a statistic could be at least those which are independent of any data. For example the number "6" in the answer below. Please note that I did not say that non-statistic estimators are necessarily bad. – IMA Jan 17 at 7:52
I think perhaps you are making too many fine distinctions which are unnecessary and, in the end, complicate your exposition. For example, "1/2" is a great estimator of the parameter of a Bernoulli variable (it is minimax for quadratic loss), so it would be a shame to rule it out just because it is independent of the data. (That would be analogous to ruling out squares as examples of rectangles in Euclidean geometry: you could do that, but that would then double the lengths of most statements concerning properties of rectangles.) It similarly helps not to rule out randomized statistics. – whuber♦ Jan 17 at 15:28
I don't think we are really talking about the same thing. Where do I rule out anything? If one-half is a great estimator, then it is a case where it is. I just don't think that the majority of possible estimators not beeing statistics are pretty great. For a Bernoulli variable "1/2" is good. But -quite- a few other estimators from the class "A real number" are not very good, wouldn't you agree? On the matter of randomized statistics still based on data- I didn't rule it out as I would still say you will require a deterministic procedure. But I concede that I should add this above. – IMA Jan 18 at 8:08
up vote 1 down vote

"6" is an example of an estimator. Say your question was, "what is the slope of the best linear function mapping x to y?" Your answer could be "6". Or it could be $(X'X)^{-1}X'Y$. Both are estimators. Which one is better is left to you to decide.

A really good TA once explained the concept of an estimator to me that way.

Basically, an estimator is a thing that you apply to data to get a quantity that you don't know the value of. You know the value of a statistic -- it is a function of the data with no "best" or "optimal" about it. There is no "best" mean. There is just a mean.

Say you have a dataset on number of goats owned per person, and each person's happiness. You're interested in how people's happiness changes with the number of goats they own. An estimator can help you to estimate that relationship from your data. Statistics are just functions of the data that you have. For example, the variance of goat ownership may equal 7. Te forula for calculating variance would be identical between goats and toasters, or whether you're interested in happiness or propensity to get cancer. In that sense, all sensible estimators are statistics.

share|improve this answer
up vote -1 down vote

In hypothesis testing :

A test-statistic is about hypothesis testing. A test-statistic is a random variable given/under the null hypothesis. Now, some may call a statistic the value/measure of the test-statistic given the sample.

With these two you can get the p-value which is a measure that helps to reject or not reject the null hypothesis. All in all, a statistic is an estimation of how far/close to your hypothesis.

This link may be useful.

share|improve this answer
You seem to be addressing a different question, something related to hypothesis tests rather than estimation. Your definition of "statistic" is much more restricted in scope than standard definitions are: statistics apply to all forms of decision making, not just the very limited cases of hypothesis testing and null hypotheses. Moreover, hypothesis tests are not the same as estimators and most statistics are not used as estimators of nearness to some hypothesis. – whuber♦ Jan 15 at 15:05
I wouldn't say it's a different question. It gives a picture about what it is in the context of hypothesis testing at least! – D.Khireche Jan 15 at 15:18
Because this answer focuses on a limited and specialized version of the question and uses the key terms "estimator" and "statistic" in unconventional ways, without alerting the reader to that fact, I worry that it may mislead or confuse people. – whuber♦ Jan 15 at 15:20
I thought Hypothesis testing was far to be a limited and specialized field of statistics. – D.Khireche Jan 15 at 15:41

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.