Intuition for expectation of discrete random variable that takes positive integers

Question

If $X$ is a discrete random variable that takes values on the positive integers, it is true that

$$E(X) = \sum_{k=1}^{\infty} P(X \ge k)\;.$$

I know how to prove this (by expressing the summand as a sum of all probabilities going from $n=k$ to $n=\infty$ and then playing around with the double sums) but I'm having trouble developing an intuition for why the above result holds.

I am thinking of a number line where the equation says that if I'm stepping through $k$ from $1$ to $\infty$ and take the sum of probabilities of everything to the right of $k$ at each step, that gives the expectation of $X$, which does not make sense to me.

This can be flipped to be $1 - \sum_{k=1}^{\infty} P(x \le k)$, whose interpretation would then be stepping though $k$ but summing the probabilities of everything to the left of $k$ at each step, but this does not help me understand either.

The result is from exercise 4.7.8 in Rice's statistics book.

Answer 1

One way to see this is as follows:

For a discrete random variable $X$, we obviously have that

$$E[X] = \sum_{k=1}^\infty kP(X=k)$$

so our goal is to see how $\sum_{k=1}^{\infty} P(X\geq k)$ relates to that. Since $X$ is a discrete rv, we also have that

$$P(X>n) = \sum_{k=n}^{\infty} P(X=k)$$

which in words is saying that $P(X>n)$ is the same as just summing every individual $P(X=k)$ from $n$ onwards. Then if we were to sum all of the $P(X>n)$ from $n=1$ to $\infty$, we have

$$\sum_{n=1}^\infty P(X>n) = \sum_{n=1}^\infty \sum_{k=n}^{\infty} P(X=k)$$

Things get confusing with this double sum, so let's take a moment to think about it intuitively. When $n=1$, we have one $P(X=k)$ for every single $k=1,\dots,\infty$. When $n=2$, we have one $P(X=k)$ for every single $k=2,\dots,\infty$. So every $P(X=k)$ gets another count, except the first one! And it's clear to see that after $n=1$, we will never count another $P(X=1)$ since the sums are $k=n,\dots,\infty$ for any $n$. So we only get one $P(X=1)$. To make things easier on our brain, let's just store them in an infinite dimensional matrix $A$ where the $(i,k)$-th element is whether $P(X=k)$ gets a count for when $n=i$ ($1$ if gets a count, $0$ otherwise). So when $n=1$ (the first row of the matrix), every $P(X=j)$ gets a count. So the first row is just an infinite vector of $1$s. For $n=2$, every $P(X=j)$ for $j\geq 2$ gets a count, so the first entry is $0$, and the rest are all $1$. For $n=3$, every $P(X=j)$ for $j \geq 3$ gets a count, so the first two entries is $0$, and the rest are all $1$.

The pattern is easy to see: for $n=i$, every $P(X=j)$ for $j \geq i$ will get a count. So let's do this till infinity, and let's count how many times we count each $P(X=j)$. For $j=1$, we only count it once. For $j=2$, we count it twice. For $j=i$, we count it $i$ times. So it must be that

$$\sum_{n=1}^\infty P(X>n) = \sum_{n=1}^\infty \sum_{k=n}^{\infty} P(X=k) = 1\times P(X=1) + 2\times P(X=2) + 3\times P(X=3) + \dots$$

But wait! That looks awfully like that first equation we wrote! And indeed, it is equal to $\sum_{k=1}^\infty P(X=k)$, which is equal to $E[X]$.

The intuition is that by counting sums of $P(X=k)$ in the way described, we are counting each one $k$ times, which is the natural way to describe the 'average' value of a discrete random variable, since this is another way to write that we are weighting each value $k$ by $P(X=k)$.