In statistics, it is very important to differentiate between the following three concepts which are often confused and mixed by students.

Usually, books denote by $\theta$ an unknown parameter. Then we wish to estimate it. We use an estimator which books usually denote by $\widehat{\theta}$. The estimator is a random variable! Usually we seek $E[\widehat{\theta}]=\theta$ and so on and on, anyways. An estimate is the value we obtain by sampling and inserting our values in our estimator.

A classical example is:

  • Parameter: population mean $\mu$.
  • Estimator $\overline{X}=\frac{1}{n}\sum_{i=1}^{n} X_i$ based on a priori observations $X_1,\dots,X_n$. And then, we sample observations $x_1,\dots,x_n$ and compute $\overline{x}=\frac{1}{n}\sum_{i=1}^{n} x_i$.
  • Obs: $\mu$ is an unknown number. $\overline{X}$ is a random variable, and $\overline{x}$ is a number!

With concrete latin letters it seems easy to stress this fact, but when we use $\theta$ and $\widehat{\theta}$ (the classical hat notation for estimator) I do not know how to stress this fact. I do not know how to differentiate between $\widehat{\theta}$ and a specific observed $\widehat{\theta}$.

Some books propose: $\widehat{\theta}_{obs}$ which I do not like for instance if we talk about two population proportions $p_1$ and $p_2$ and their estimators $\widehat{p}_1$ and $\widehat{p}_2$. Because then it would look like $\widehat{p}_{1,obs}$ which is not aesthetic.

What solutions do you propose? Has anyone seen a nice notation for this?


There is no single answer to this question because different authors may use different notation. For me, the most handy notation is the one used, for example, by Larry Wasserman in All of Statistics:

By convention, we denote a point estimate of $\theta$ by $\hat\theta$ or $\widehat\theta_n$. Remember that $\theta$ is a fixed, unknown quantity. The estimate $\hat\theta$ depends on the data so $\hat\theta$ is a random variable.

More formally, let $X_1,\dots,X_n$ be $n$ iid data points from some distribution $F$. A point estimator $\widehat\theta_n$ of a parameter $\theta$ is some function of $X_1,\dots,X_n$:

$$ \widehat\theta_n = g(X_1,\dots,X_n). $$

So $\theta$ is the unknown parameter, $\hat\theta$ is the estimate, and a function $g$ of the sample is the estimator. Such notation makes it also clear that $g$ is a function.

| cite | improve this answer | |
  • $\begingroup$ Sorry, did you mean, $\widehat{\theta}_n$ is the estimator (a random variable) and $\widehat{\theta}$ is the estimate (a number)? $\endgroup$ – Martingalo Aug 2 '18 at 10:05
  • 1
    $\begingroup$ @Martingalo, no I meant what I've said $\hat\theta$ (or alternatively $\hat\theta_n$) are estimates, $g$ is estimator. $\endgroup$ – Tim Aug 2 '18 at 10:09
  • $\begingroup$ aah I see, aha.. mmm.. these are students who struggle a lot, and I guess introducing functions of random variables would be too complicated, but sure it is a good alternative :) Thank you! $\endgroup$ – Martingalo Aug 2 '18 at 10:44
  • $\begingroup$ @Martingalo but estimators are functions, I can't see how you could not mention it... $\endgroup$ – Tim Aug 2 '18 at 10:50
  • $\begingroup$ You wouldn't like to know how I define estimators ;) it's an extremely low level course. I just say estimators are "algorithms/formulas" that are applied to samples, more or less. I want to avoid the word function, because this belongs to mathematics which they barely know. Hence, my question. But nevermind :) $\endgroup$ – Martingalo Aug 2 '18 at 11:28

You are right that the use of lower-case Greek letters creates a potential ambiguity here; this is a common issue in teaching estimation theory to students. To differentiate between an estimator and the corresponding estimate I find it helpful to use notation that includes the data as an argument value, and thereby stresses that we have a function of the data:

$$\begin{matrix} \text{Estimator} & & & \hat{\theta}(\boldsymbol{X}), \\[6pt] \text{Estimate } \text{ } & & & \hat{\theta}(\boldsymbol{x}). \\[6pt] \end{matrix}$$

With this notation you subsitute the upper case $\boldsymbol{X}$ to denote the random variable (estimator) and the lower-case $\boldsymbol{x}$ to denote a fixed observed value (estimate). It also has the advantage of being more technically sound, since for a fixed $n$ the estimator is a function $\hat{\theta}: \mathscr{X}^n \rightarrow \Theta$. So when I am writing with this notation I usually say things like this:

For large $n$ we can rely on the central limit theorem to write the distribution of the estimator as $\hat{\theta}(\boldsymbol{X}) \sim \text{N}(\theta, \sigma_{se}^2)$. In our previous simulation example we simulated values from a distribution with true mean $\theta = 3$, yielding data $\boldsymbol{x} = (3.1, 5.2, 1.6)$, giving us the estimate $\hat{\theta}(\boldsymbol{x}) = 3.3$.

Once the students understand the difference between the random estimator and the fixed estimate, and if the meaning is obvious from context, you can then drop the argument later.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.