Why does a Cumulative Distribution Function (CDF) uniquely define a distribution?

Question

I have always been told a CDF is unique however a PDF/PMF is not unique, why is that ? Can you give an example where a PDF/PMF is not unique ?

Answer 1

Let us recall some things. Let $(\Omega,A,P)$ be a probability space, $\Omega$ is our sample set, $A$ is our $\sigma$-algebra, and $P$ is a probability function defined on $A$. A random variable is a measurable function $X:\Omega \to \mathbb{R}$ i.e. $X^{-1}(S) \in A$ for any Lebesgue measurable subset in $\mathbb{R}$. If you are not familiar with this concept then everything I say afterwards will not make any sense.

Anytime we have a random variable, $X:\Omega \to \mathbb{R}$, it induces a probability measure $X'$ on $\mathbb{R}$ by the categorical pushforward. In other words, $X'(S) = P(X^{-1}(S))$. It is trivial to check that $X'$ is probability measure on $\mathbb{R}$. We call $X'$ the distribution of $X$.

Now related to this concept is something called the distribution function of a function variable. Given a random variable $X:\Omega \to \mathbb{R}$ we define $F(x) = P(X\leq x)$. Distribution functions $F:\mathbb{R} \to [0,1]$ have the following properties:

1. $F$ is right-continuous.

2. $F$ is non-decreasing

3. $F(\infty) = 1$ and $F(-\infty)=0$.

Clearly random variables which are equal have the same distribution and distribution function.

To reverse the process and obtain a measure with the given distribution function is pretty technical. Let us say you are given a distribution function $F(x)$. Define $\mu(a,b] = F(b) - F(a)$. You have to show that $\mu$ is a measure on the semi-algebra of intervals of the $(a,b]$. Afterwards you can apply the Carathéodory extension theorem to extend $\mu$ to a probability measure on $\mathbb{R}$.

Answer 2

To answer the request for an example of two densities with the same integral (i.e. have the same distribution function) consider these functions defined on the real numbers:

 f(x) = 1 ; when x is odd integer
 f(x) = exp(-x^2)  ; elsewhere

and then;

 f2(x) = 1  ; when x is even integer
 f2(x) = exp(-x^2) ;  elsewhere

They are not equal at all x, but are both densities for the same distribution, hence densities are not uniquely determined by the (cumulative) distribution. When densities with a real domain are different only on a countable set of x values, then the integrals will be the same. Mathematical analysis is not really for the faint of heart or the determinately concrete mind.

Answer 3

I disagree with the statement, "the probability distribution function does not uniquely determine a probability measure", that you say in your opening question. It does uniquely determine it.

Let $f_1,f_2:\mathbb{R}\to [0,\infty)$ be two probability mass functions. If, $$ \int_E f_1 = \int_E f_2 $$ For any measurable set $E$ then $f_1=f_2$ almost everywhere. This uniquely determines the pdf (because in analysis we do not care if they disagree on a set of measure zero).

We can rewrite the above integral into, $$ \int_E g = 0 $$ Where $g=f_1-f_2$ is an integrable function.

Define $E = \{ x \in \mathbb{R} ~ | ~ g \geq 0 \}$, so $\int_E g = 0$. We use the well-known theorem that if an integral of a non-negative function is zero then the function is zero almost everywhere. In particular, $g=0$ a.e. on $E$. So $f_1 = f_2$ a.e. on $E$. Now repeat the argument in the other direction with $F = \{ x\in \mathbb{R} ~ | ~ g \leq 0 \}$. We will get that $f_1 = f_2$ a.e on $F$. Thus, $f_1 = f_2$ a.e. on $E\cup F = \mathbb{R}$.