# Help with a proof regarding empirical CDF

Given $$X_1,X_2...X_n \sim F$$ and $$(\hat{F}_n(x))$$ is an empirical CDF

I need to prove that $$Var(\hat{F}_n(x)) = \frac{F(x)(1 - F(x))}{n}$$

So what I did is:

The variance of an estimator $$\hat{F}_n(x)$$ of a population CDF $$F(x)$$ is this: (if I'm not mistaken)

$$Var(\hat{F}_n(x)) = E[(\hat{F}_n(x) - E[\hat{F}_n(x)])^2]$$

Since $$\hat{F}_n(x)$$ is an estimator of the population CDF $$F(x)$$, we can substitute $$F(x)$$ for $$E[\hat{F}_n(x)]$$ in the above equation to get:

$$Var(\hat{F}_n(x)) = E[(\hat{F}_n(x) - F(x))^2]$$

We can then use the fact that $$\hat{F}_n(x)$$ is a Bernoulli random variable with probability of success equal to $$F(x)$$ to expand the square and simplify the expression:

$$Var(\hat{F}_n(x)) = E[\hat{F}_n(x)^2 - 2\hat{F}_n(x)F(x) + F(x)^2]$$

Since it's a bernoulli we get that $$\hat{F}_n(x)^2 = \hat{F}_n(x)$$ and we know that $$E[\hat{F}_n(x)] = F(x)$$, therefore we plug the new values:

$$Var(\hat{F}_n(x)) = F(x) - 2F(x)^2 + F(x)^2 = F(x)(1 - F(x))$$

And this is where I get to a dead end, I get this formula but what I need to prove is this formula but with the right side divided by $$n$$, and I didn't get this. What did I do wrong?

# EDIT-

First we find the variance of $$\mathop{\hat{F}_n}\left(x\right)$$.

The variance of a random variable is defined as $$Var(X) = E[(X - E[X])^2]$$.

Substituting $$\mathop{\hat{F}_n}\left(x\right)$$ for $$X$$, we have:

$$Var(\hat{F}_n(x)) = E[(\hat{F}_n(x) - E[\hat{F}_n(x)])^2]$$

We can rewrite $$\hat{F}_n(x)$$ as $$\frac{R_n(x)}{n}$$, where $$R_n(x) \sim \text{Binomial}(n, F(x))$$, and that is because empirical CDF is the average of Bernoulli random variables. Therefore,

$$Var(\hat{F}_n(x)) = E\left[\left(\frac{R_n(x)}{n} - E\left[\frac{R_n(x)}{n}\right]\right)^2\right]$$

$$Var(\hat{F}_n(x)) = E\left[\frac{R_n(x)^2}{n^2}\right] - \left(E\left[\frac{R_n(x)}{n}\right]\right)^2$$

Since $$R_n(x)$$ is a binomial random variable, we can use the formula for the variance of a binomial random variable, which is $$Var(R_n(x)) = nF(x)(1 - F(x))$$. Therefore,

$$Var(\hat{F}_n(x)) = \frac{nF(x)(1 - F(x))}{n^2} - \left(\frac{nF(x)}{n}\right)^2$$

After simplifying we get:

$$Var(\hat{F}_n(x)) = \frac{F(x)(1 - F(x))}{n}$$

• Recall the definition of the ecdf, which is an average of Bernoulli r.v.s Jan 3, 2023 at 10:43
• Stating that "$\hat{F}_n(x)$ is an estimator of the population CDF $F(x)$" is not enough to conclude that "we can substitute $F(x)$ for $E[\hat{F}_n(x)]$". Jan 3, 2023 at 11:00
• @ChristophHanck So what should I add to my solution? I'm missing the point.
– CORy
Jan 3, 2023 at 11:28
• Basically what the answers that have come in by now say - recall that $Var(aX)=a^2Var(X)$. Jan 3, 2023 at 13:11
• Because $n\hat F(x)$ counts how many independent copies of $X$ are in the interval $(-\infty, x],$ it has a Binomial distribution with parameters $n$ and $F(x)$ and you're done.
– whuber
Jan 3, 2023 at 16:51

Define $$Y_i(x)=\mathbb I_{\{X_i\leq x\}}$$ $$\forall i\in\{1, 2,\ldots, n\}.$$

Notice $$Y_i(x) \overset{\text{iid}}{\sim}\mathcal{Ber}(\theta)\tag 1\label 1$$ where $$\theta := F(x) .$$

Now express (how?) $$n \hat F_n(x) =\sum_{i=1}^n Y_i(x) ;\tag 2$$

Use $$\eqref 1$$ above to yield $$\operatorname{Var}(F_n(x)).$$

• concise and straight to the point (+1). Trivial note: the numbering (2) seems redundant, it is not actually used. Jan 3, 2023 at 21:10
• I numbered that for if OP needed, I would have expanded. Jan 3, 2023 at 22:16

Note that you can write $$\mathop{\hat{F}_n}\left(x\right)$$ as $$\mathop{\hat{F}_n}\left(x\right) = \mathop{R_n}\left(x\right)/n$$, where $$\mathop{R_n}\left(x\right) \sim \mathop{\text{Binomial}}\left(n, \mathop{F}\left(x\right)\right)$$.

Proof.

$$\mathop{R_n}\left(x\right) \mathrel{:=}\sum_{i=1}^n \mathop{\mathbf{1}_{\left(-\infty,\, x\right]}}\left(X_i\right)$$ counts the number of successes, meaning the number of $$X_i$$s in $$\left(-\infty, x\right]$$, in $$n$$ independent Bernoulli trials with success probability $$\mathop{\mathbb{P}}\left(X_i \in \left(-\infty, x\right] \right) = \mathop{F}\left(x\right)$$ each. Hence, $$\mathop{R_n}\left(x\right) \sim \mathop{\text{Binomial}}\left(n, \mathop{F}\left(x\right)\right)$$.
By definition, $$\mathop{\hat{F}_n}\left(x\right) = n^{-1}\sum_{i=1}^n \mathop{\mathbf{1}_{\left(-\infty,\, x\right]}}\left(X_i\right)$$ and the statement follows.

• That is $R_n(x)$ here?
– CORy
Jan 3, 2023 at 11:26
• @CORy Hint: It's a sum of $n$ i.i.d. indicator random variables. Jan 3, 2023 at 11:30
• So maybe $\hat{F}_n(x) = 1/n \sum i$ where i is the random variable?
– CORy
Jan 3, 2023 at 11:58
• @CORy what does your random variable $i$ stand for? Jan 3, 2023 at 12:03
• @CORy You already use $\text{Var}(R_n(x)) = nF(x)(1 - F(x))$ in your edited derivation. What does this tell you about $\text{Var}(R_n(x)/n)$? Do you need the other variance decomposition steps? Jan 3, 2023 at 12:40