2
$\begingroup$

This example demonstrates the difference between a theoretical observation and a realized observation. A theoretical observation is a random variable with a probability distribution, while its realization is a number. Some statistical concepts have different word concept from its realization. For example, an estimator is a random variable, while an estimate is it realization. But this is an exception. Most statistical concepts - and 'observation' is one of them - have only one word to denote both. In the previous chapter, observations were realized (given the data), but now they denote theoretical (random) data.

This an excerpt from an econometrics book. I would like to focus on the estimator part. I need two clarifications.

  1. I am not happy now because the answer of @whuber here What is the relation between estimator and estimate? seems to contradict with this excerpt which says an estimator is a random variable. According to this excerpt, as I interpret it, the OLS estimator is a random variable because it depends on X and y, which are random, and hence it has a sampling distribution. Therefore it has a variance (that we estimate). OLS estimator is a statistic. This thread also claims that the estimator is a random variable Why is an estimator considered a random variable?

  2. Given what is above, is it true that when talking about the variance, we should always talk about the variance of the estimator and not the variance of the estimate? An estimate is not a random variable. But if you search google, you will find countless hits that talk about the "variance of the OLS estimate".

It may help this thread if we keep the discussion within the bounds of econometrics and not get into very complex mathematical perspectives.

Improve this question
$\endgroup$
7
  • $\begingroup$ If you really believe this quotation, then please test it out. Since it claims an "estimator is a random variable," then (a) write down a familiar estimator and (b) see whether you can demonstrate, through basic definitions, that it is a random variable. For instance, let's consider the most common estimator: that of a mean. Given a sequence of $n$ numbers $x_i$, the mean is defined as $\sum_{i=1}^n x_i/n.$ Can you go from there (without invoking any additional assumptions!) to showing how this is some kind of random variable? $\endgroup$
    – whuber ♦
    Oct 17, 2018 at 14:09
  • $\begingroup$ I already proposed the OLS estimator. It depends on X and y which are random variables and hence the estimator is a random variable. Since it is a random variable it has a sampling distribution. It has an expectation and a variance. And that is why we are talking about the `statistical properties' of the OLS estimator. $\endgroup$
    – Snoopy
    Oct 17, 2018 at 14:33
  • $\begingroup$ You are mixing up several things. It's not a problem, but there is a conceptual advantage to keeping them distinct, as seen in the better expositors of the theory (Lehmann, Kiefer, et al.). The OLS estimator is a function $t$ (which takes a matrix $X$ and vector $y$ and returns $(X^\prime X)^{-}X^\prime y$). On its face, this has nothing to do with random variables. However, when you compose $t$ with a random variable $(X,y)$ or with fixed $X$ and a random variable $y,$ then this composition itself becomes a random variable. The statistical properties depend on those of $(X,y).$ $\endgroup$
    – whuber ♦
    Oct 17, 2018 at 17:31
  • $\begingroup$ @whuber Okay, we shall refer to a composition. Given that we always consider such a composition, the OLS estimator is a random variable. This is what I conclude from this explanation. My second question remains to be addressed. $\endgroup$
    – Snoopy
    Oct 17, 2018 at 18:11
  • $\begingroup$ @whuber The case you make for the sample mean refers to the OLS "estimate". Given data, an estimate is not a random variable. So I do not understand the point in the initial reply. $\endgroup$
    – Snoopy
    Oct 18, 2018 at 10:24

1 Answer 1

Reset to default
-1
$\begingroup$

1) @whuber may disagree but I think there are some semantics at play here. An estimator is a definition whereas an estimate is a value. Now, without any context, an estimator isn’t a true random variable because it has no bounds (the sample mean is an estimator of the population mean). Without further information it can’t be considered a true RV.

With a bit more context it can be conceived as such: ‘the mean height of a sample of people is an estimate of population heights’. Now it is a random variable taking on possible values of human heights.

Finally, the act of taking a sample and computing a mean gives rise to a realized estimate, the sample mean.

2) you should talk about the variance of the estimator for expository purposes but the variance of the estimate for practical purposes.

Improve this answer
$\endgroup$
1
  • $\begingroup$ I can't tell whether I agree or disagree because some key points are vague. First, because random variables aren't required to be bounded, it's not evident what you're claiming by stating something is "not a true random variable." Second, the distinction you make between "expository" and "practical" needs some motivation and explanation: is there a difference between the two meanings of "variance" or not? $\endgroup$
    – whuber ♦
    Oct 17, 2018 at 14:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.