Notation for ECDF

Question

I'm reviewing statistical functionals and U-statistics, trying to make notes, and I am tripping on notation. From my understanding, $X$ is used to denote a random variable and $x$ is used to denote an observation or realization of $X$.

Essentially, imagine n i.i.d. random variables such that $X_1, ..., X_n \stackrel{i.i.d}{\sim} F$. Now, I'm seeing the ECDF defined as $$\hat{F}_{n}(x) = \frac{1}{n} \sum_{i=1}^{n} \mathbb{I}(X_i \leq x) $$ The ECDF is determined using an observed sample so why isn't this denoted instead as, $${F}_{n}(x) = \frac{1}{n} \sum_{i=1}^{n} \mathbb{I}(X_i \leq x) $$ and then, given realizations of $X_1, ..., X_n$, $$\hat{F}_{n}(x) = \frac{1}{n} \sum_{i=1}^{n} \mathbb{I}(x_i \leq x) $$

What am I missing here?

Answer 1

The "hat" denotes estimation generally --- it does not distinguish the estimator from the estimate

In statistical notation, when we have an estimator/estimate of an unknown object, it is usual to denote the estimator/estimate with the same symbol as the object being estimated, but with a "hat" over the top to denote estimation. Moreover, if we want to refer to an estimator/estimate using a specific number of data points, we usually put the number of data points as a subscript. Thus, if we have an unknown distribution $F$ then the notation $\hat{F}_n$ is a standard way to refer to an estimator/estimate of that function, using $n$ data points.

In your question, you appear to be proposing to use the "hat" to distinguish the estimator from the estimate, which is not what this notational object is usually used for. In most statistical discussions, this distinction is made using the standard convention for using upper or lower-case letters to distinguish random variables from constants. Unfortunately, in the present case, this conflicts with the convention of using an upper-case letter to denote a CDF, and this latter convention prevails over the former. It is possible to distinguish the estimator from the estimate even in this case, and I will show you below how this would usually be done. However, the "hat" is not used to make this distinction.

---

Distinguishing between the estimator and estimate in this problem: This is a situation where your ECDF function is implicitly dependent on other objects not specified explicitly as arguments, and this is what is leading to confusion. Let's look again at the stated definition of the ECDF:

$$\hat{F}_{n}(x) \equiv \frac{1}{n} \sum_{i=1}^{n} \mathbb{I}(X_i \leq x) \quad \quad \quad \text{for all } x \in \mathbb{R}.$$

A cumulative distribution function gives you a probability value for each real input, so the function itself is an element of the set $[0,1]^\mathbb{R}$. However, you can see here that in addition to being dependent on the argument $x$, the output of this function also depends on the random sample. Technically, the output of this function is dependent on three things: the real argument $x \in \mathbb{R}$, the random vector $\mathbf{X}_n = (X_1,...,X_n)$ (which is a mapping $\mathbf{X}_n: \Omega \rightarrow \mathbb{R}^n$), and the realised value $\omega \in \Omega$ in the sample space. In the above definition we are choosing not to include explicit dependence on the latter two arguments, but let's rewrite this with notation that refers to all arguments. Taking the random vector $\mathbf{X}_n$ as a fixed mapping, we get the function:

$$\hat{F}_{n}(x, \omega| \mathbf{X}_n) \equiv \frac{1}{n} \sum_{i=1}^{n} \mathbb{I}(X_i(\omega) \leq x) \quad \quad \quad \text{for all } x \in \mathbb{R} \text{ and } \omega \in \Omega.$$

With this new notation we are including all arguments explicitly, and we now see that the ECDF depends on the random vector that is our sample, and the outcome in the sample space. You can now distinguish between the estimator (which is a random variable) and the estimate (which is the output you get with your observed data)

$$\begin{matrix} \text{Estimator} & & & \hat{F}_{n}(\ \cdot , \cdot \ | \mathbf{X}_n): \Omega \rightarrow [0,1]^\mathbb{R}, \\[6pt] \text{Estimate} & & & \hat{F}_{n}(\ \cdot , \omega | \mathbf{X}_n) \in [0,1]^\mathbb{R}. \quad \ \ \\[6pt] \end{matrix}$$

Here we see that the estimator is a random CDF, whereas the estimate is a fixed CDF. As you can see, all of this is a bit of a mouthful, and the notation gets a bit technical, which is why statistical exposition of estimators tends to gloss over this part, and sometimes uses the same notation for estimators and their corresponding estimates. While the above gives a fully technically correct way of expressing things, a simpler shorthand is just to show both objects as a function of the data, and distinguish them using upper and lower-case for the input data, like this:

$$\begin{matrix} \text{Estimator (at } x \text{)} & & & \hat{F}_{n}(x |\mathbf{X}_n), \ \\[6pt] \text{Estimate (at } x \text{)} & & & \hat{F}_{n}(x |\mathbf{x}_n).\ \\[6pt] \end{matrix}$$

---

Sometimes capitalisation helps, and sometimes it doesn't: Of course, when conventions for capitalisation and non-capitalisation permit (i.e., when they are not in conflict), we can draw a notational distinction between these objects using the standard convention of writing the random version (estimator) in upper-case and the fixed version (estimate) in lower-case. In many contexts this is what is done --- e.g., when using the sample mean as an estimator of the true mean we would commonly use $\bar{X}_n$ and $\bar{x}_n$ as the respective notations for the estimator and the estimate.

However, in the present case, use of upper/lower case conflicts with the corresponding convention for capitalisation that distinguishes a CDF from a density function --- i.e., the letter $F$ refers here to the underlying CDF of the data, and $f$ would usually refer to its density/mass function. Thus, if we were to use $\hat{f}_n$ here to denote the estimated CDF (i.e., the estimate), that notation would be confusing to the reader, because it would look like we were talking about an estimator for the density $f$. As you can see, this is a case where two contradictory conventions for capitalisation occurs, and so we choose for the CDF/density distinction to prevail over the random variable/fixed value distinction.

Nevertheless, as you can see from the above technical argument, even in cases where conventions for capitalisation make it problematic to use upper-case and lower-case letters to distinguish the estimator from the estimate, it is possible to distinguish the two by making the implicit arguments explicit. This allows us to make a notational distinction between the estimator and estimate if we wish to do so.

Answer 2

Comment continued with some graphics.

Suppose you have a sample of size $n = 200$ from the distribution $\mathsf{Gamma}(\mathrm{shape} = 5, \mathrm{rate} = 1/10,$ so that the distribution mean $\mu = 50.$

Here are such data simulated in R:

set.seed(1234)
x = rgamma(200, 5, 0.1)
summary(x)
  Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
 11.00   35.04   47.99   50.67   61.02  145.84 

Here is a histogram of the data (blue bars) with the density of $\mathsf{Gamma}(5, 0.1)$ (black curve), and the default KDE from R (red dotted curve).

hist(prob=T, br=15, xlim=c(0,150), xaxs="i", col="skyblue2", 
      main = "n=200: GAMMA(5, .1)")
rug(x)  # tick marks along horizontal axis
curve(dgamma(x, 5, .1), add=T, lwd=2)
lines(density(x), type="l", col="red", lwd=2, lty="dotted")

Here is an ECDF of the data x (blue), along with the CDF of $\mathsf{Gamma}(5, 0.1).$

plot(ecdf(x), col="skyblue", 
     main="ECDF of Sample (blue) and CDF of GAMMA(5, .1)")
 curve(pgamma(x, 5, .1), add=T, lwd=2)