Notation in statistics (parameter/estimator/estimate)

Question

In statistics, it is very important to differentiate between the following three concepts which are often confused and mixed by students.

Usually, books denote by $\theta$ an unknown parameter. Then we wish to estimate it. We use an estimator which books usually denote by $\widehat{\theta}$. The estimator is a random variable! Usually we seek $E[\widehat{\theta}]=\theta$ and so on and on, anyways. An estimate is the value we obtain by sampling and inserting our values in our estimator.

A classical example is:

• Parameter: population mean $\mu$.
• Estimator $\overline{X}=\frac{1}{n}\sum_{i=1}^{n} X_i$ based on a priori observations $X_1,\dots,X_n$. And then, we sample observations $x_1,\dots,x_n$ and compute $\overline{x}=\frac{1}{n}\sum_{i=1}^{n} x_i$.
• Obs: $\mu$ is an unknown number. $\overline{X}$ is a random variable, and $\overline{x}$ is a number!

With concrete latin letters it seems easy to stress this fact, but when we use $\theta$ and $\widehat{\theta}$ (the classical hat notation for estimator) I do not know how to stress this fact. I do not know how to differentiate between $\widehat{\theta}$ and a specific observed $\widehat{\theta}$.

Some books propose: $\widehat{\theta}_{obs}$ which I do not like for instance if we talk about two population proportions $p_1$ and $p_2$ and their estimators $\widehat{p}_1$ and $\widehat{p}_2$. Because then it would look like $\widehat{p}_{1,obs}$ which is not aesthetic.

What solutions do you propose? Has anyone seen a nice notation for this?

Answer 1

There is no single answer to this question because different authors may use different notation. For me, the most handy notation is the one used, for example, by Larry Wasserman in All of Statistics:

By convention, we denote a point estimate of $\theta$ by $\hat\theta$ or $\widehat\theta_n$. Remember that $\theta$ is a fixed, unknown quantity. The estimate $\hat\theta$ depends on the data so $\hat\theta$ is a random variable.

More formally, let $X_1,\dots,X_n$ be $n$ iid data points from some distribution $F$. A point estimator $\widehat\theta_n$ of a parameter $\theta$ is some function of $X_1,\dots,X_n$:

$$ \widehat\theta_n = g(X_1,\dots,X_n). $$

So $\theta$ is the unknown parameter, $\hat\theta$ is the estimate, and a function $g$ of the sample is the estimator. Such notation makes it also clear that $g$ is a function.

Answer 2

You are right that the use of lower-case Greek letters creates a potential ambiguity here; this is a common issue in teaching estimation theory to students. To differentiate between an estimator and the corresponding estimate I find it helpful to use notation that includes the data as an argument value, and thereby stresses that we have a function of the data:

$$\begin{matrix} \text{Estimator} & & & \hat{\theta}(\boldsymbol{X}), \\[6pt] \text{Estimate } \text{ } & & & \hat{\theta}(\boldsymbol{x}). \\[6pt] \end{matrix}$$

With this notation you substitute the upper case $\boldsymbol{X}$ to denote the random variable (estimator) and the lower-case $\boldsymbol{x}$ to denote a fixed observed value (estimate). It also has the advantage of being more technically sound, since for a fixed $n$ the estimator is a function $\hat{\theta}: \mathscr{X}^n \rightarrow \Theta$. So when I am writing with this notation I usually say things like this:

For large $n$ we can rely on the central limit theorem to write the distribution of the estimator as $\hat{\theta}(\boldsymbol{X}) \sim \text{N}(\theta, \sigma_{se}^2)$. In our previous simulation example we simulated values from a distribution with true mean $\theta = 3$, yielding data $\boldsymbol{x} = (3.1, 5.2, 1.6)$, giving us the estimate $\hat{\theta}(\boldsymbol{x}) = 3.3$.

Once the students understand the difference between the random estimator and the fixed estimate, and if the meaning is obvious from context, you can then drop the argument later.

Answer 3

I was using the letter $g$ to denote estimators (e.g., $g_\theta$ or $g_{\mu_X}$), but I became concerned because the letter $g$ is widespread to denote Hedge's $g$ in meta-analyses. I believe there is no single letter in the English alphabet that isn't already commonly used to denote something else in statistics, much less in mathematics broadly. This is not necessarily a deal-breaker for the use of the letter $g$, but we could use a more descriptive string of characters to denote the function, much like how statisticians may use $\operatorname{var}(\theta)$ to denote $\sigma^2_\theta$, $\operatorname{cov}(X,Y)$ to denote $\sigma_{X,Y}$, $\operatorname{Bias}(g)$ to denote the bias of an estimator $g$, etc. Thus, here is a suggestion:

parameter: $\theta$

estimator of $\theta$: $\operatorname{estr}_\theta$

estimate of $\theta$: $\hat\theta$

standard error of $\operatorname{estr}_\theta$: $\sigma_{\operatorname{estr}_\theta}$

estimator of $\sigma_{\operatorname{estr}_\theta}$: $\operatorname{estr}_{\sigma_{\operatorname{estr}_\theta}}$

However, a major disadvantage of putting the estimand into the subscript is that it is likely too small for readers, especially when considering standard errors and estimators of standard errors.

As far as I am aware (I also asked ChatGPT 4), "estr" is not an operator name that is already used in other areas of mathematics, thus there is little to no risk of cross-disciplinary confusion with using it.

Notice the letter 'r' in $\operatorname{estr}$ is important, otherwise it would be $\operatorname{est}$ which novices may easily misinterpret as "estimate".

Note: I have a prior similar answer on this page, which I updated to reflect a major limitation of that one.

Answer 4

EDIT: I was endorsing the notation indicated below. At first I was on board, on the basis of the fact that the notation for probabilities (e.g., $\mathbb{P}(\theta)$) also appears function-like but doesn't actually follow the function-like convention of putting inside the parentheses the argument(s) and/or operand(s). However, I'm concerned it could be too confusing. Thus, I have a new suggested notation indicated in another answer I've posted on this page.

Original post:

parameter: $\theta$

estimator of $\theta$: $\operatorname{estr}\left(\theta\right)$

estimate of $\theta$: $\hat\theta$

standard error of $\operatorname{estr}\left(\theta\right)$: $\sigma_{\operatorname{estr}\left(\theta\right)}$

estimator of the standard error of $\operatorname{estr}\left(\theta\right)$: $\operatorname{estr}\left(\sigma_{\operatorname{estr}\left(\theta\right)}\right)$

As far as I am aware (I also asked ChatGPT 4), "estr" is not an operator name that is already used in other areas of mathematics, thus there is little to no risk of cross-disciplinary confusion with using it.

Notice the letter 'r' in $\operatorname{estr}()$ is important, otherwise it would be $\operatorname{est}()$ which novices may easily misinterpret as "estimate"