$\newcommand{\A}{\mathcal{A}}
\newcommand{\B}{\mathcal{B}}
\newcommand{\C}{\mathbb{C}}
\newcommand{\E}{\mathbf{E}}
\newcommand{\F}{\mathcal{F}}
\newcommand{\G}{\mathcal{G}}
\renewcommand{\H}{\mathcal{H}}
\renewcommand{\L}{\mathcal{L}}
\newcommand{\M}{\mathcal{M}}
\newcommand{\N}{\mathbf{N}}
\renewcommand{\P}{\mathbf{P}}
\newcommand{\Q}{\mathbb{Q}}
\newcommand{\R}{\mathbf{R}}
\newcommand{\s}{\mathbf{S}}
\newcommand{\W}{\mathbb{W}}
\newcommand{\X}{\mathcal{X}}
\newcommand{\Z}{\mathbb{Z}}
\newcommand{\al}{\alpha}
\newcommand{\be}{\beta}
\newcommand\arrowp{\stackrel{P}{\to}}
\newcommand\arrowd{\stackrel{d}{\to}}
\newcommand{\d}{\mathop{}\!\mathrm{d}}$
In this answer (to my own question), I show how the elementary definition of an expectation of a continuous random variable is consistent with the abstract measure theoretic formulation, and also mention the appearance of the Radon-Nikodym derivative. I heavily use notation from David William's Probability with Martingales [PW], and also restate his Lemma from Section 6.12 along with results from Section 5.14 in [PW] with detailed proof.
Definitions
- Let $(\Omega,\F,\P)$ be a probability space, where $\Omega$ is a non-empty set, $\F$ is a $\sigma$-algebra on $\Omega$, and $\P\colon\F\to \R$ is a probability measure.
- Let $\B$ be the Borel $\sigma$-algebra on $\R$.
- Let $m\F$ be the set of random variables, i.e., $$m\F=\{X\colon\Omega\to\R:X\text{ is }\F/\B\text{ measurable}\}.$$
- Let $m\B$ be the set of real-valued measurable functions on $\R$, i.e. $$m\B=\{h\colon\R\to\R:h\text{ is }\B/\B\text{ measurable}\}.$$
- Let $\Lambda_X$ be the law of $X$, which is the probability measure $\Lambda_X\colon\B\to\R$ such that $\Lambda_X(B)=\P(X\in B)$ for all $B\in\B$.
- Use $\E$ to denote integration of elements of $m\F$ with respect to the measure $\P$, e.g. $\E[X]=\int_{\Omega} X \d \P$.
- Let $\mathbf{L}_X$ be the integration operator (analagous to $\E$) of elements of $m\B$ with respect to $\Lambda_X$. That is, $\mathbf{L}_X(h)=\int_{\R} h\d\Lambda_X$ for each $h\in m\B$, where integration is with respect to $\Lambda_X$ instead of $\P$.
- Set $\L^1(\Omega,\F,\P) = \{X \in m\F:\E|X|<\infty\},$ the set of integrable random variables with respect to $\P$.
- Let $\L^1(\R,\B,\Lambda_{X})= \{h\in m\B: \int_{\R}|h|\d\Lambda_X<\infty \}$, the set of integrable functions on $\R$ with respect to $\Lambda_X$.
- Let $\mathrm{Leb}$ denote the Lebesgue measure, and we write $\int_{\R} h \d\mathrm{Leb}$ or $\int_{\R}h(x)\d x$ to denote the Lebesgue integral of $h\in m\B$.
Note that we are using two different notions of integral for $\E$ and $\Lambda_X$. That is, just as we define integration of $\F/\B$ measurable functions $X\colon\Omega\to\R$ with respect to the probability measure $\P$, we may define integration of $\B/\B$ measurable functions $h\colon\R\to\R$ with respect to the probability measure $\Lambda_X$. Instead of $\int_{\Omega} X\d \P$, we write $\int_{\R} h \d \Lambda_X$. Observe that $\mathbf{L}_X(h)$ relates to $X$ precisely through the measure $\Lambda_X$ we are integrating with respect to, while $\P$ has no connection a priori to $X$ in $\E[h(X)]$.
Our goal is to establish that the $\mathbf{L}_X$ and $\E$ operators agree (Proposition 1), i.e., $\int_{\R} h\d\Lambda_X = \int_{\Omega}h(X)\d\P$, and in the special case that the random variable admits a density, $\mathbf{L}_X$ has the standard elementary form $\int_{\R}xf(x)\d x$ (Proposition 3).
Equivalence of Integration with respect to Law $\Lambda_X$ and with respect to $\P$
We state and prove the Lemma from Section 6.2 of [PW]. It states that a $\B/\B$-measurable function $h$ is integrable with respect to $\Lambda_X$ iff $h(X)$ is integrable with respect to $\P$, in which case the integrals agree. More compactly,
Proposition 1: Suppose $h\in m\B$. Then $h(X)\in\L^1(\Omega,\F,\P)$ iff $h\in\L^1(\R,\B,\Lambda_X)$ and $\E h(X)= \mathbf{L}_X(h)$, i.e. $\int_{\R} h\d\Lambda_X = \int_{\Omega}h(X)\d\P$.
Proof.
First, suppose $h=1_B$ for $B\in\B$. Then $$\E h(X)=\E[1_{\{X\in B\}}] =P(X\in B),$$ and $$\mathbf{L}_X(h) =\int_{\R}1_B \d\Lambda_X = \int_B \d\Lambda_X = \Lambda_X(B) :=P(X\in B).$$
Linearity of abstract integration will demonstrate that equality holds for each simple function.
Next, let $h$ be a non-negative function in $m\B$. Then there is a sequence $(h_n)$ of non-negative simple functions in $m\B$ such that $h_n\uparrow h$. By the Monotone Convergence Theorem applied to each distinct measure space, $$\E[h(X)]=\lim_n \E[h_n(X)] = \lim_n \mathbf{L}_X(h_n)=\mathbf{L}_X(h),$$ since $\E[h_n(X)]=\mathbf{L}_X(h_n)$ for each $n$.
Most generally, if $h\in m\B$, write $h=h^{+}-h^{-}$ where $h^{+}=\max(0,h)$ and $h^{-}=\max(-h,0)$ and apply linearity.
When a PDF Exists
Next, let's obtain a familiar expression for $\mathbf{L}_X(h)$ when $X$ admits a PDF $f_X$, i.e., $$\Lambda_X(B)=P(X\in B)=\int_B f_X\d \mathrm{Leb} = \int_B f_X(x)\d x.$$ In the notation of [PW], this means $\Lambda_X$ has measure $f_X\mathrm{Leb}$, $\Lambda_X$ has density $f_X$ relative to $\mathrm{Leb}$, and we write $\frac{\d \Lambda_X}{\d \mathrm{Leb}}=f_X$. The PDF $f_X$ is called the Radon-Nikodym derivative of $\Lambda_X$ relative to $\mathrm{Leb}$ on $(\R,\B)$. Our precise formulation is below, but more suggestively, we demonstrate that $$\int_{\R} h(x)f_X(x)\d x=\int_{\R} h f_X\d\mathrm{Leb} = \int_{\R} h \frac{\d \Lambda_X}{\d \mathrm{Leb}}\d\mathrm{Leb} = \int_{\R} h\d\Lambda_X.$$
Proposition 2: Suppose $h\in m\B$. Then $h(X)\in\L^1(\Omega,\F,\P)$ iff $h\in\L^1(\R,\B,\Lambda_X)$. Assume $h\in \L^1(\R,\B,\Lambda_X)$ where $\Lambda_X$ has density $f_X$ relative to $\mathrm{Leb}$. Then $$\int_{\R} h\d\Lambda_X =\int_{\R}hf_X\d\mathrm{Leb}.$$
Proof.
First, suppose $h=1_C$ where $C\in\B$. Then $$
\int_{\R}h\d\Lambda_X = \int_C \d\Lambda_X =\Lambda_X(C),$$ and $$
\int_{\R}hf_X\d\mathrm{Leb} = \int_C f_X\d\mathrm{Leb} =P(X\in C)=\Lambda_X(C).$$ Our assertion immediately extends to simple functions using linearity properties of both integrals.
Suppose $h\in\L^1(\R,\B,\Lambda_X)$ with $h\ge0$. Then there is an increasing sequence $(h_n)$ of non-negative simple functions such that $h_n\uparrow h$. By the Monotone Convergence Theorem applied to each distinct measure space, \begin{align*}
\int_{\R}h\d\Lambda_X =\lim_n\int_{\R}h_n\d\Lambda_X=\lim_n\int_{\R} h_nf_X\d\mathrm{Leb}= \int_{\R} hf_X\d\mathrm{Leb},
\end{align*} yielding the claim. For general $h\in\L^1(\R,\B,\Lambda_X)$, writing $h=h^{+}-h^{-1}$ finishes the proof.
The following final result combines Proposition 2 with Proposition 1 to yield $$\int_{\Omega} h(X)\d \P=\int_{\R} h(x)f_X(x)\d x,$$ where the latter integral is the Lebesgue integral.
Proposition 3: Suppose $h\in m\B$. Then $h(X)\in\L^1(\Omega,\F,\P)$ iff $h\in\L^1(\R,\B,\Lambda_X)$. Moreover, if $\Lambda_X$ has density $f_X$ relative to $\mathrm{Leb}$, then $$\E[X]=\int_{\Omega} h(X)\d \P =\int_{\R}hf_X\d\mathrm{Leb}.$$
References
Williams, D. (1991). Probability with Martingales. Cambridge: Cambridge University Press. doi:10.1017/CBO9780511813658
