3
$\begingroup$

Let $(\Omega,\mathcal{F},P)$ be a probability space, let $X\colon \Omega\to\mathbb{R}$ be real-valued and measurable. Suppose there exists $f\colon \mathbb{R}\to [0,\infty]$ such that $P(X\in A)=\int_A f(x)\mathrm{d}x$ for each $A\in\mathrm{Borel}(\mathbb{R})$.

I wish to show that $\mathbb{E}[X]=\int X\mathrm{d}P=\int_{\mathbb{R}}xf(x)\mathrm{d}x$, using the most elementary techniques possible (e.g. using indicator and simple functions, followed by integral limit theorems, as opposed to Radon-Nikodym, pushforward measures, etc.) Other answers I have found seem to use technical reasoning I am unfamiliar with.

One result I have proven is the following: Define $\mu\colon\mathrm{Borel}(\mathbb{R})\to[0,1]$ by $\mu(A)=P(X\in A)$. Then for any measurable function $g\colon\mathbb{R}\to[0,\infty]$, $$\mathbb{E}[g(X)]=\int_{\mathbb{R}}g \mathrm{d}\mu.$$

If we set $g=\mathrm{id}$ in the previous claim, then we obtain $\mathbb{E}[X]=\int_{\mathbb{R}}\mathrm{d}\mu$. Can we use the fact that $\mu(A)=\int_A f(x)\mathrm{d}x$ for each $A\in\mathrm{Borel}(\mathbb{R})$, to write $\int_{\mathbb{R}} \mathrm{d}\mu=\int_{\mathbb{R}}xf(x)\mathrm{d}x$.

$\endgroup$
1
  • 2
    $\begingroup$ Your formula for the expectation is wrong. If $X$ has a density $f$, then $EX=\int xf(x)\, dx=\int x dF(x)$. $\endgroup$ May 5, 2019 at 22:37

1 Answer 1

1
$\begingroup$

$\newcommand{\A}{\mathcal{A}} \newcommand{\B}{\mathcal{B}} \newcommand{\C}{\mathbb{C}} \newcommand{\E}{\mathbf{E}} \newcommand{\F}{\mathcal{F}} \newcommand{\G}{\mathcal{G}} \renewcommand{\H}{\mathcal{H}} \renewcommand{\L}{\mathcal{L}} \newcommand{\M}{\mathcal{M}} \newcommand{\N}{\mathbf{N}} \renewcommand{\P}{\mathbf{P}} \newcommand{\Q}{\mathbb{Q}} \newcommand{\R}{\mathbf{R}} \newcommand{\s}{\mathbf{S}} \newcommand{\W}{\mathbb{W}} \newcommand{\X}{\mathcal{X}} \newcommand{\Z}{\mathbb{Z}} \newcommand{\al}{\alpha} \newcommand{\be}{\beta} \newcommand\arrowp{\stackrel{P}{\to}} \newcommand\arrowd{\stackrel{d}{\to}} \newcommand{\d}{\mathop{}\!\mathrm{d}}$

In this answer (to my own question), I show how the elementary definition of an expectation of a continuous random variable is consistent with the abstract measure theoretic formulation, and also mention the appearance of the Radon-Nikodym derivative. I heavily use notation from David William's Probability with Martingales [PW], and also restate his Lemma from Section 6.12 along with results from Section 5.14 in [PW] with detailed proof.

Definitions

  • Let $(\Omega,\F,\P)$ be a probability space, where $\Omega$ is a non-empty set, $\F$ is a $\sigma$-algebra on $\Omega$, and $\P\colon\F\to \R$ is a probability measure.
  • Let $\B$ be the Borel $\sigma$-algebra on $\R$.
  • Let $m\F$ be the set of random variables, i.e., $$m\F=\{X\colon\Omega\to\R:X\text{ is }\F/\B\text{ measurable}\}.$$
  • Let $m\B$ be the set of real-valued measurable functions on $\R$, i.e. $$m\B=\{h\colon\R\to\R:h\text{ is }\B/\B\text{ measurable}\}.$$
  • Let $\Lambda_X$ be the law of $X$, which is the probability measure $\Lambda_X\colon\B\to\R$ such that $\Lambda_X(B)=\P(X\in B)$ for all $B\in\B$.
  • Use $\E$ to denote integration of elements of $m\F$ with respect to the measure $\P$, e.g. $\E[X]=\int_{\Omega} X \d \P$.
  • Let $\mathbf{L}_X$ be the integration operator (analagous to $\E$) of elements of $m\B$ with respect to $\Lambda_X$. That is, $\mathbf{L}_X(h)=\int_{\R} h\d\Lambda_X$ for each $h\in m\B$, where integration is with respect to $\Lambda_X$ instead of $\P$.
  • Set $\L^1(\Omega,\F,\P) = \{X \in m\F:\E|X|<\infty\},$ the set of integrable random variables with respect to $\P$.
  • Let $\L^1(\R,\B,\Lambda_{X})= \{h\in m\B: \int_{\R}|h|\d\Lambda_X<\infty \}$, the set of integrable functions on $\R$ with respect to $\Lambda_X$.
  • Let $\mathrm{Leb}$ denote the Lebesgue measure, and we write $\int_{\R} h \d\mathrm{Leb}$ or $\int_{\R}h(x)\d x$ to denote the Lebesgue integral of $h\in m\B$.

Note that we are using two different notions of integral for $\E$ and $\Lambda_X$. That is, just as we define integration of $\F/\B$ measurable functions $X\colon\Omega\to\R$ with respect to the probability measure $\P$, we may define integration of $\B/\B$ measurable functions $h\colon\R\to\R$ with respect to the probability measure $\Lambda_X$. Instead of $\int_{\Omega} X\d \P$, we write $\int_{\R} h \d \Lambda_X$. Observe that $\mathbf{L}_X(h)$ relates to $X$ precisely through the measure $\Lambda_X$ we are integrating with respect to, while $\P$ has no connection a priori to $X$ in $\E[h(X)]$.

Our goal is to establish that the $\mathbf{L}_X$ and $\E$ operators agree (Proposition 1), i.e., $\int_{\R} h\d\Lambda_X = \int_{\Omega}h(X)\d\P$, and in the special case that the random variable admits a density, $\mathbf{L}_X$ has the standard elementary form $\int_{\R}xf(x)\d x$ (Proposition 3).

Equivalence of Integration with respect to Law $\Lambda_X$ and with respect to $\P$

We state and prove the Lemma from Section 6.2 of [PW]. It states that a $\B/\B$-measurable function $h$ is integrable with respect to $\Lambda_X$ iff $h(X)$ is integrable with respect to $\P$, in which case the integrals agree. More compactly,


Proposition 1: Suppose $h\in m\B$. Then $h(X)\in\L^1(\Omega,\F,\P)$ iff $h\in\L^1(\R,\B,\Lambda_X)$ and $\E h(X)= \mathbf{L}_X(h)$, i.e. $\int_{\R} h\d\Lambda_X = \int_{\Omega}h(X)\d\P$.

Proof. First, suppose $h=1_B$ for $B\in\B$. Then $$\E h(X)=\E[1_{\{X\in B\}}] =P(X\in B),$$ and $$\mathbf{L}_X(h) =\int_{\R}1_B \d\Lambda_X = \int_B \d\Lambda_X = \Lambda_X(B) :=P(X\in B).$$

Linearity of abstract integration will demonstrate that equality holds for each simple function.

Next, let $h$ be a non-negative function in $m\B$. Then there is a sequence $(h_n)$ of non-negative simple functions in $m\B$ such that $h_n\uparrow h$. By the Monotone Convergence Theorem applied to each distinct measure space, $$\E[h(X)]=\lim_n \E[h_n(X)] = \lim_n \mathbf{L}_X(h_n)=\mathbf{L}_X(h),$$ since $\E[h_n(X)]=\mathbf{L}_X(h_n)$ for each $n$.

Most generally, if $h\in m\B$, write $h=h^{+}-h^{-}$ where $h^{+}=\max(0,h)$ and $h^{-}=\max(-h,0)$ and apply linearity.


When a PDF Exists

Next, let's obtain a familiar expression for $\mathbf{L}_X(h)$ when $X$ admits a PDF $f_X$, i.e., $$\Lambda_X(B)=P(X\in B)=\int_B f_X\d \mathrm{Leb} = \int_B f_X(x)\d x.$$ In the notation of [PW], this means $\Lambda_X$ has measure $f_X\mathrm{Leb}$, $\Lambda_X$ has density $f_X$ relative to $\mathrm{Leb}$, and we write $\frac{\d \Lambda_X}{\d \mathrm{Leb}}=f_X$. The PDF $f_X$ is called the Radon-Nikodym derivative of $\Lambda_X$ relative to $\mathrm{Leb}$ on $(\R,\B)$. Our precise formulation is below, but more suggestively, we demonstrate that $$\int_{\R} h(x)f_X(x)\d x=\int_{\R} h f_X\d\mathrm{Leb} = \int_{\R} h \frac{\d \Lambda_X}{\d \mathrm{Leb}}\d\mathrm{Leb} = \int_{\R} h\d\Lambda_X.$$


Proposition 2: Suppose $h\in m\B$. Then $h(X)\in\L^1(\Omega,\F,\P)$ iff $h\in\L^1(\R,\B,\Lambda_X)$. Assume $h\in \L^1(\R,\B,\Lambda_X)$ where $\Lambda_X$ has density $f_X$ relative to $\mathrm{Leb}$. Then $$\int_{\R} h\d\Lambda_X =\int_{\R}hf_X\d\mathrm{Leb}.$$

Proof. First, suppose $h=1_C$ where $C\in\B$. Then $$ \int_{\R}h\d\Lambda_X = \int_C \d\Lambda_X =\Lambda_X(C),$$ and $$ \int_{\R}hf_X\d\mathrm{Leb} = \int_C f_X\d\mathrm{Leb} =P(X\in C)=\Lambda_X(C).$$ Our assertion immediately extends to simple functions using linearity properties of both integrals.

Suppose $h\in\L^1(\R,\B,\Lambda_X)$ with $h\ge0$. Then there is an increasing sequence $(h_n)$ of non-negative simple functions such that $h_n\uparrow h$. By the Monotone Convergence Theorem applied to each distinct measure space, \begin{align*} \int_{\R}h\d\Lambda_X =\lim_n\int_{\R}h_n\d\Lambda_X=\lim_n\int_{\R} h_nf_X\d\mathrm{Leb}= \int_{\R} hf_X\d\mathrm{Leb}, \end{align*} yielding the claim. For general $h\in\L^1(\R,\B,\Lambda_X)$, writing $h=h^{+}-h^{-1}$ finishes the proof.


The following final result combines Proposition 2 with Proposition 1 to yield $$\int_{\Omega} h(X)\d \P=\int_{\R} h(x)f_X(x)\d x,$$ where the latter integral is the Lebesgue integral.


Proposition 3: Suppose $h\in m\B$. Then $h(X)\in\L^1(\Omega,\F,\P)$ iff $h\in\L^1(\R,\B,\Lambda_X)$. Moreover, if $\Lambda_X$ has density $f_X$ relative to $\mathrm{Leb}$, then $$\E[X]=\int_{\Omega} h(X)\d \P =\int_{\R}hf_X\d\mathrm{Leb}.$$

References

Williams, D. (1991). Probability with Martingales. Cambridge: Cambridge University Press. doi:10.1017/CBO9780511813658

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.