2
$\begingroup$

I have an empirical distribution $S_n(x)$ (= proportion of samples less than equal to x) from a random sample $X_1, X_2, ..., X_n$ for a random variable $X \sim F_X$. Consider the random variable $T_n(x) := n.S_n(x)$. This is a binomial random variable $B(n, F_X(x))$.

What is the expectation $E(T_n(x)T_n(y))$ when $x < y$?

Reference: Corollary 2.3.13, Nonparametric Statistical Inference by Gibbons-Chakraborti, Fifth edition.

They have given the answer as $nF_X(x) + n(n-1)F_X(x)F_X(y)$.

I have tried to write it as a product of indicator variables, $E(I(X_i \le x)I(X_i \le y))$ but I'm unable to complete the derivation. I appreciate hints.

Edit :

With @whuber's hint, I separate the expectation into two cases, when $i = j$ and when $i \ne j$:

$E(T_n(x)T_n(y)) = E(I(X_i \le x)I(X_j \le y))$

$E(T_n(x)T_n(y)) = E(I(X_i \le x)I(X_i \le y)) + E(I(X_i \le x)I(X_j \le y))$

The product is one if and only if both the indicators are one.

In the first term, as $x < y$, $I(X_i \le x)I(X_i \le y) = I(X_i \le x)$ as the latter indicator is a superset of the former. This has an expectation $nF_X(x)$.

In the second term, as the samples are independent, I can write the expectation as product of expectations, $E(I(X_i \le x)I(X_j \le y)) = E(I(X_i \le x))E(I(X_j \le y))$.

Now, the first expectation is similar to the first case. But, why is the second expectation $(n-1)F_X(y)$?

$\endgroup$
2
  • 2
    $\begingroup$ Clearly -- correcting a typo -- you need to compute $E[I(X_i\le x)I(X_j\le y)].$ Because the sample is random, the $X_i$ are exchangeable, implying there are just two such expectations to compute: one where $i=j$ and the other where $i\ne j.$ Where do you run into problems computing them? $\endgroup$
    – whuber
    Commented Sep 2, 2023 at 16:02
  • $\begingroup$ @whuber Thanks a lot! I see my mistake, will try with this. $\endgroup$
    – Mewbacca
    Commented Sep 2, 2023 at 16:16

1 Answer 1

2
$\begingroup$

The basis of this result is the observation that

$$F(x) = \Pr(X_i\le x) = E[I(X_i\le x)]$$

for all $i.$

For $i\ne j,$ the variables $X_i$ and $X_j$ are independent, whence so are their indicators, giving

$$E[I(X_i\le x)I(X_j\le y)] = E[I(X_i\le x)]E[I(X_j\le y)] = F(x)F(y).$$

When $i=j,$ because $x\lt y$ implies $I(X_i\le x)I(X_i\le y)= I(X_i\le \min(x,y))=I(X_i\le x),$ it follows

$$E[I(X_i\le x)I(X_i\le y)] = E[I(X_i\le x)] = F(x).$$

Consequently, because $T_n(x) = I(X_1\le x)+I(X_2\le x)+\cdots + I(X_n\le x)$ counts the number of $X_i$ less than or equal to $x,$ linearity of expectation gives

$$\begin{aligned} E[T_n(x)T_n(y)]&=E\left[\sum_{i=1}^n \sum_{j=1}^n I(X_i\le x)I(X_j\le y)\right]\\ &= \sum_{i=1}^n E[I(X_i \le x)I(X_i\le x)] + \sum_{i\ne j}^n E[I(X_i\le x)I(X_j\le x)]\\ &= \sum_{i=1}^n F(x) + \sum_{i\ne j}F(x)F(y)\\ &= nF(x) + n(n-1)F(x)F(y). \end{aligned}$$

The coefficient $n$ counts the number of terms in the double sum where $i=j$ while the coefficient $n(n-1)=n^2-n$ counts the remaining terms in the double sum where $i\ne j.$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.