19
$\begingroup$

I have always been told a CDF is unique however a PDF/PMF is not unique, why is that ? Can you give an example where a PDF/PMF is not unique ?

$\endgroup$
  • 6
    $\begingroup$ Concerning uniqueness, you might like to ponder the difference between the PDF of a uniform distribution on $[0,1]$ and a uniform distribution on its interior, $(0,1)$. Another fun exercise--which addresses the question of whether a PDF even exists--is to think about what the PDF of a distribution over the rational numbers would look like. For instance, let $\Pr(j2^{-i})=2^{1-2i}$ whenever $0\lt j2^{-i}\lt 1$, $i\ge 1$, and $j$ is odd. $\endgroup$ – whuber Feb 6 '15 at 23:43
  • 2
    $\begingroup$ Not all distributions even have a PDF, or have a PMF, while looking at the CDF gives a unifying view to things. Continuous variables have smooth-looking CDFs, discrete variables have a "staircase", and some CDFs are mixed. $\endgroup$ – Silverfish Feb 7 '15 at 0:01
  • 6
    $\begingroup$ @Silverfish: ...and some are none of the above! :-) $\endgroup$ – cardinal Feb 7 '15 at 1:17
  • 3
    $\begingroup$ To address the title (perhaps somewhat loosely), the CDF defines a distribution because the CDF (or equivalently just DF/'distribution function'; the "C" acts only to clarify that's the object we're talking about) is what the term 'distribution' literally refers to; the "D" is the clue on that part. That it's unique follows from the "F" -- functions are single-valued, so if two distribution functions are identical the object they define is the same; if the DFs differed anywhere the thing they are the definition of would be different at those points. Is that tautology? I think it is. $\endgroup$ – Glen_b Feb 7 '15 at 2:31
  • 4
    $\begingroup$ @Glen_b It's tautological only to the trained intuition. A distribution function $F$ only gives probabilities of the form $F(x)=\Pr\{\omega\in\Omega\,|\,X(\omega)\le x\}$ whereas the entire distribution specifies probabilities of the form $\Pr(\{\omega\in\Omega\,|\,X(\omega)\in\mathcal{B}\}$ for arbitrary measurable sets $\mathcal{B}\subset\mathbb R$. You have to show $F$ determines the distribution. As NicholasB points out, that's a matter of extending a pre-measure from a semi-ring (of half-open intervals), $\mu((a,b])=F(b)-F(a)$, to the full Lebesgue sigma-field and showing it's unique. $\endgroup$ – whuber Feb 10 '15 at 20:51
14
$\begingroup$

Let us recall some things. Let $(\Omega,A,P)$ be a probability space, $\Omega$ is our sample set, $A$ is our $\sigma$-algebra, and $P$ is a probability function defined on $A$. A random variable is a measurable function $X:\Omega \to \mathbb{R}$ i.e. $X^{-1}(S) \in A$ for any Lebesgue measurable subset in $\mathbb{R}$. If you are not familiar with this concept then everything I say afterwards will not make any sense.

Anytime we have a random variable, $X:\Omega \to \mathbb{R}$, it induces a probability measure $X'$ on $\mathbb{R}$ by the categorical pushforward. In other words, $X'(S) = P(X^{-1}(S))$. It is trivial to check that $X'$ is probability measure on $\mathbb{R}$. We call $X'$ the distribution of $X$.

Now related to this concept is something called the distribution function of a function variable. Given a random variable $X:\Omega \to \mathbb{R}$ we define $F(x) = P(X\leq x)$. Distribution functions $F:\mathbb{R} \to [0,1]$ have the following properties:

  1. $F$ is right-continuous.

  2. $F$ is non-decreasing

  3. $F(\infty) = 1$ and $F(-\infty)=0$.

Clearly random variables which are equal have the same distribution and distribution function.

To reverse the process and obtain a measure with the given distribution function is pretty technical. Let us say you are given a distribution function $F(x)$. Define $\mu(a,b] = F(b) - F(a)$. You have to show that $\mu$ is a measure on the semi-algebra of intervals of the $(a,b]$. Afterwards you can apply the Carathéodory extension theorem to extend $\mu$ to a probability measure on $\mathbb{R}$.

$\endgroup$
  • 4
    $\begingroup$ This is a good start to an answer, but may be unintentionally obscuring the matter at hand a bit. The main issue seems to be showing that two measures with the same distribution function are, in fact, equal. This requires nothing more than Dynkin's $\pi $-$\lambda $ theorem and the fact that sets of the form $(-\infty, b] $ form a $\pi $-system that generates the Borel $\sigma $-algebra. Then the nonuniqueness of a density (assuming it exists!) can be addressed and contrasted with the above. $\endgroup$ – cardinal Feb 10 '15 at 19:37
  • 3
    $\begingroup$ (One additional minor quibble: Random variables are usually defined in terms of Borel sets rather than Lebesgue sets.) I think with some minor edits this answer will become quite clear. :-) $\endgroup$ – cardinal Feb 10 '15 at 19:38
  • $\begingroup$ @cardinal I think of analysis first, probability second. Therefore, this may explain why I prefer to think of Lebesgue sets. In either case it does not affect what was said. $\endgroup$ – Nicolas Bourbaki Feb 10 '15 at 20:17
4
$\begingroup$

To answer the request for an example of two densities with the same integral (i.e. have the same distribution function) consider these functions defined on the real numbers:

 f(x) = 1 ; when x is odd integer
 f(x) = exp(-x^2)  ; elsewhere

and then;

 f2(x) = 1  ; when x is even integer
 f2(x) = exp(-x^2) ;  elsewhere

They are not equal at all x, but are both densities for the same distribution, hence densities are not uniquely determined by the (cumulative) distribution. When densities with a real domain are different only on a countable set of x values, then the integrals will be the same. Mathematical analysis is not really for the faint of heart or the determinately concrete mind.

$\endgroup$
0
$\begingroup$

I disagree with the statement, "the probability distribution function does not uniquely determine a probability measure", that you say in your opening question. It does uniquely determine it.

Let $f_1,f_2:\mathbb{R}\to [0,\infty)$ be two probability mass functions. If, $$ \int_E f_1 = \int_E f_2 $$ For any measurable set $E$ then $f_1=f_2$ almost everywhere. This uniquely determines the pdf (because in analysis we do not care if they disagree on a set of measure zero).

We can rewrite the above integral into, $$ \int_E g = 0 $$ Where $g=f_1-f_2$ is an integrable function.

Define $E = \{ x \in \mathbb{R} ~ | ~ g \geq 0 \}$, so $\int_E g = 0$. We use the well-known theorem that if an integral of a non-negative function is zero then the function is zero almost everywhere. In particular, $g=0$ a.e. on $E$. So $f_1 = f_2$ a.e. on $E$. Now repeat the argument in the other direction with $F = \{ x\in \mathbb{R} ~ | ~ g \leq 0 \}$. We will get that $f_1 = f_2$ a.e on $F$. Thus, $f_1 = f_2$ a.e. on $E\cup F = \mathbb{R}$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.