Expectation for a function of a discrete random variable

Question

This is a bit on the elementary side, but I'm having trouble understanding if the following formula is a definition, or a derivation.

Given that $E[X] = \sum\limits_x xp(x)$, and $g(X)$ is some real-valued function, then $E[g(X)] = \sum\limits_xg(x)p(x)$.

From an intuitive standpoint, this makes perfect sense. Finding the expected value of a function of $X$ is fundamentally no different than finding the expectation for $X$ itself, as the expectation is just the theoretical average of all possible values of $g$.

But is this something that

• we define explicitly,
• follows directly from the definition of $E[X]$, or
• something that can be proven to follow from the formula for $E[X]$?

The reason I ask is that I understand $E[X]$ and $E[g(X)]$ in isolation, but I'm having trouble determining if the bridge between them is mathematically provable, or if the latter is just an "obvious" definition.

It also seems that most proofs for properties of expectation (such as linearity; $E[aX + b] = aE[X] +b$) begins with the knowledge that $E[g(X)] = \sum\limits_xg(x)p(x)$ so it's pretty fundamental.

Answer 1

This is not a definition; it is the discrete version of the law of the unconcious statistician. To derive the law, suppose we start with some discrete random variable $X$ which ranges over the domain $\mathscr{X}$. Now let $S = g(X)$ be our transformed discrete random variable and let $\mathscr{S} \equiv \{ g(x) | x \in \mathscr{X} \}$ be the domain of this latter random variable.

To facilitate our analysis, define the sets $\mathscr{X}(s) \equiv \{ x \in \mathscr{X} | g(x) = s \}$ for all $s \in \mathscr{S}$. It is trivial to show that these sets form a partition of the space $\mathscr{X}$ and give:

$$\begin{align} \mathbb{P}(S=s) &= \sum_{x \in \mathscr{X}(s)} \mathbb{P}(X=x). \\[6pt] \end{align}$$

We therefore have:

$$\begin{align} \mathbb{E}(g(X)) &= \mathbb{E}(S) \\[6pt] &= \sum_{s \in \mathscr{S}} s \cdot \mathbb{P}(S=s) \\[6pt] &= \sum_{s \in \mathscr{S}} s \sum_{x \in \mathscr{X}(s)} \mathbb{P}(X=x) \\[6pt] &= \sum_{s \in \mathscr{S}} \sum_{x \in \mathscr{X}(s)} s \cdot \mathbb{P}(X=x) \\[6pt] &= \sum_{s \in \mathscr{S}} \sum_{x \in \mathscr{X}(s)} g(x) \cdot \mathbb{P}(X=x) \\[6pt] &= \sum_{x \in \mathscr{X}} g(x) \cdot \mathbb{P}(X=x). \\[6pt] \end{align}$$

Answer 2

Suppose that $X$ is a discrete random variable taking on distinct values $u_1, u_2, \cdots$ with probabilities $p_1,p_2, \cdots, $ respectively. Then, the definition of the expected value $E[X]$ of $X$ is $$E[X] = \sum_i u_i P\{X = u_i\} = \sum_i u_ip_i\tag{1}$$ Now, suppose that there is a random variable $Y = g(X)$ where $g(\cdot)$ is some real function. Then, whenever $X$ takes on value $u_i$, $Y$ takes on value $g(u_i)$. Thus, $Y$ is also a discrete random variable that takes on values $v_1, v_2, \cdots$ where each $v_j$ equals $g(u_i)$ for some not necessarily unique $i$, that is, depending in what $g(\cdot)$ is, there might be more than one solution to $g(u) = v_j$.

For example, suppose that $X$ takes on $5$ distinct values $u_1, u_2, \cdots, u_5$; and $g(\cdot)$ is such that $g(u_2)$ and $g(u_5)$ both have value $v_1$, $g(u_1)$ has value $v_2$, while $g(u_3)$ and $g(u_4)$ both have value $v_3$, that is, $Y$ only takes on three distinct values $v_1, v_2, v_3$. Furthermore, it must be the case that \begin{alignat}{4} P\{Y = v_1\} &= P\{X = u_2\} + P\{X=u_5\} &=\, &p_2+p_5\\ P\{Y = v_2\} &= P\{X = u_1\} &= &\,p_1\\ P\{Y = v_3\} &= P\{X = u_3\} + P\{X=u_4\} &=\, &p_3+p_4. \end{alignat} Applying the definition of expected value, we have that \begin{align}\require{cancel} E[Y] &= \sum_j v_jP\{Y=v_j\}\\ &= v_1(p_2+p_5) + v_2(p_1) + v_3(p_3+p_4)\\ &= \cancelto{g(u_2)}{v_1}p_2 + \cancelto{g(u_5)}{v_1}p_5 + \cancelto{g(u_1)}{v_2}p_1 + \cancelto{g(u_3)}{v_3}p_3 + \cancelto{g(u_4)}{v_3}p_4\\ &= \sum_i g(u_i)p_i\\ &= \sum_i g(u_i)P\{X=u_i\}\tag{2} \end{align} which is LOTUS, the law of the unconscious statistician.

Anyway, that's my intuition as to why LOTUS is true. A formal proof would need to talk of an onto mapping from the set $\{u_i\}$ to the (possibly smaller) set $\{v_j\}$ etc.