What is the relation between estimator and estimate?

Question

What is the relation between estimator and estimate?

Answer 1

E. L. Lehmann, in his classic Theory of Point Estimation, answers this question on pp 1-2.

The observations are now postulated to be the values taken on by random variables which are assumed to follow a joint probability distribution, $P$, belonging to some known class...

...let us now specialize to point estimation...suppose that $g$ is a real-valued function defined [on the stipulated class of distributions] and that we would like to know the value of $g$ [at whatever is the actual distribution in effect, $\theta$]. Unfortunately, $\theta$, and hence $g(\theta)$, is unknown. However, the data can be used to obtain an estimate of $g(\theta)$, a value that one hopes will be close to $g(\theta)$.

In words: an estimator is a definite mathematical procedure that comes up with a number (the estimate) for any possible set of data that a particular problem could produce. That number is intended to represent some definite numerical property ($g(\theta)$) of the data-generation process; we might call this the "estimand."

The estimator itself is not a random variable: it's just a mathematical function. However, the estimate it produces is based on data which themselves are modeled as random variables. This makes the estimate (thought of as depending on the data) into a random variable and a particular estimate for a particular set of data becomes a realization of that random variable.

In one (conventional) ordinary least squares formulation, the data consist of ordered pairs $(x_i, y_i)$. The $x_i$ have been determined by the experimenter (they can be amounts of a drug administered, for example). Each $y_i$ (a response to the drug, for instance) is assumed to come from a probability distribution that is Normal but with unknown mean $\mu_i$ and common variance $\sigma^2$. Furthermore, it is assumed that the means are related to the $x_i$ via a formula $\mu_i = \beta_0 + \beta_1 x_i$. These three parameters--$\sigma$, $\beta_0$, and $\beta_1$--determine the underlying distribution of $y_i$ for any value of $x_i$. Therefore any property of that distribution can be thought of as a function of $(\sigma, \beta_0, \beta_1)$. Examples of such properties are the intercept $\beta_0$, the slope $\beta_1$, the value of $\cos(\sigma + \beta_0^2 - \beta_1)$, or even the mean at the value $x=2$, which (according to this formulation) must be $\beta_0 + 2 \beta_1$.

In this OLS context, a non-example of an estimator would be a procedure to guess at the value of $y$ if $x$ were set equal to 2. This is not an estimator because this value of $y$ is random (in a way completely separate from the randomness of the data): it is not a (definite numerical) property of the distribution, even though it is related to that distribution. (As we just saw, though, the expectation of $y$ for $x=2$, equal to $\beta_0 + 2 \beta_1$, can be estimated.)

In Lehmann's formulation, almost any formula can be an estimator of almost any property. There is no inherent mathematical link between an estimator and an estimand. However, we can assess--in advance--the chance that an estimator will be reasonably close to the quantity it is intended to estimate. Ways to do this, and how to exploit them, are the subject of estimation theory.

Answer 2

In short: an estimator is a function and an estimate is a value that summarizes an observed sample.

An estimator is a function that maps a random sample to the parameter estimate:

$$ \hat{\Theta}=t(X_1,X_2,...,X_n) $$ Note that an estimator of n random variables $X_1,X_2,...,X_n$ is a random variable $\hat{\Theta}$. For instance, an estimator is the sample mean: $$ \overline{X}=\frac{1}{n}\sum_{n=1}^nX_i $$ An estimate $\hat{\theta}$ is the result of applying the estimator function to a lowercase observed sample $x_1,x_2,...,x_n$:

$$ \hat{\theta}=t(x_1,x_2,...,x_n) $$ For instance, an estimate of the observed sample $x_1,x_2,...,x_n$ is the sample mean: $$ \hat{\mu}=\overline{x}=\frac{1}{n}\sum_{n=1}^nx_i $$

Answer 3

It might be helpful to illustrate whuber's answer in the context of a linear regression model. Let's say you have some bivariate data and you use Ordinary Least Squares to come up with the following model:

Y = 6X + 1

At this point, you can take any value of X, plug it into the model and predict the outcome, Y. In this sense, you might think of the individual components of the generic form of the model (mX + B) as estimators. The sample data (which you presumably plugged into the generic model to calculate the specific values for m and B above) provided a basis on which you could come up with estimates for m and B respectively.

Consistent with @whuber's points in our thread below, whatever values of Y a particular set of estimators generate you for are, in the context of linear regression, thought of as predicted values.

(edited -- a few times -- to reflect the comments below)

Answer 4

Suppose you received some data, and you had some observed variable called theta. Now your data can be from a distribution of data, for this distribution, there is a corresponding value of theta that you infer which is a random variable. You can use the MAP or mean for calculating the estimate of this random variable whenever the distribution of your data changes. So the random variable theta is known as an estimate, a single value of the unobserved variable for a particular type of data.

While estimator is your data, which is also a random variable. For different types of distributions you have different types of data and thus you have a different estimate and thus this corresponding random variable is called the estimator.