2
$\begingroup$

I wanted to see if I understood the concept of discrete random variables correctly. Therefore I want to make some statements that I believe are true only for discrete random variables and wanted to ask if these statements are true:

1.) A discrete RV X can have an uncountable range but only a countable number of these points are associated with preimages that have nonzero probability

2.) A discrete RV X can have an uncountable sample space but only a countable number of these points have a probability that is not zero.

3.) A discrete RV X does not have a density (pdf) w.r.t. the Lebesgue measure

4.) A discrete RV X can have a density w.r.t. other measures such as the counting measure

5.) The cdf F of a discrete RV is discrete (i.e., it is not continuous)

My questions are 1.) Are of these statements correct 2.) Are statemtns 1,2,3,5 equivalent and can they all be used interchangeably to define discrete random variables?

$\endgroup$
3
$\begingroup$

Although these statements capture the idea pretty well, we should take some care concerning their logical status: some provide definitions; others are characterizations not unique to discrete random variables; and a couple of them are meaningless (depending on how broadly they are interpreted).

It can be useful to bear in mind some examples of discrete random variables. In addition to Bernoulli variables (which assign nonzero probabilities to just two values) and Poisson variables (which assign nonzero probabilities to a subset of a countable lattice of values), there are discrete variables defined on dense subsets of the reals.

Examples of non-discrete variables are also useful. The classic one is a Cantor variable. Its cdf is almost everywhere constant--a property enjoyed by all discrete variables--but this is not a discrete variable.


Definitions

Many writers appear to use the following characterization as if it were a definition:

"A random variable $X:\Omega\to S$ is said to be discrete if $S$ is finite or countable" Probability, Bard College

"Let $Y$ be a metric space. A $Y$ −valued random variable is called discrete if its range is a countable set ..." Measure Theory, Mark Dean, Brown University.

These are deficient as definitions for two reasons. First, neither states explicitly what it means by "range" and this term has two common meanings in mathematics: as the set of values the function can have (its image) or the set in which its values all must lie (its codomain). It has a third meaning in probability.

The second reason is that regardless of how you understand "range," there are plenty of discrete random variables that do not have countable images.

Let's fix up these problems with the following definition. I will state it only for random variables with real values; those concerned about random variables with other kinds of values (in abstract metric spaces, for instance) will know how to generalize it.

Definition. A real-valued random variable $X$ defined on a probability space $(\Omega,\mathfrak F, \mathbb P)$ is discrete when there exists a countable subset $S\subset\mathbb R$ for which $\mathbb{P}(X \in S) = 1.$

In other words, a discrete variable almost surely has a value within a limited set of possibilities.


Implications

Let's go through the five statements of the question.

  1. "A discrete RV $X$ can have an uncountable range but only a countable number of these points are associated with preimages that have nonzero probability."

    This does not define discrete variables. Although it can be true of a discrete random variable, it is also true of non-discrete variables. As a practical example, consider a "zero inflated lognormally" distributed variable $X:$ this is a mixture of a lognormal variable (with continuous distribution) and an atom at zero. The preimage of $0$ has nonzero probability and $0$ is the only number with this property, showing $X$ satisfies property $(1)$ but obviously is not discrete.

  2. "A discrete RV $X$ can have an uncountable sample space but only a countable number of these points have a probability that is not zero."

    This is meaningless. There does not exist any probability space for which more than a countable number of points have a nonzero probability, because (by virtue of the sigma-additivity of the measure) the space would then have infinite probability rather than total probability of $1.$

  3. "A discrete RV $X$ does not have a density (pdf) w.r.t. the Lebesgue measure" $\lambda.$

    Although this does not define discrete variables, it is true of them, because the contrapositive statement is readily proven by contradiction: when $X$ has a density $f_X,$ by definition that means for every real number $x,$ $$\Pr(X\le x) = \int_{-\infty}^x f_X(x)\,\mathrm{d}\lambda(x).$$ Let $S$ be a countable subset as in the definition of "discrete" above. Since $\lambda(S)=0,$ removing $S$ from the range of integration will not change any integral with respect to Lebesgue measure $\lambda,$ whence

    $$1 = \Pr(X\in\mathbb R) = \int_{\mathbb R} f_X(x)\,\mathrm{d}\lambda(x) = \int_{\mathbb R\setminus S} f_X(x)\,\mathrm{d}\lambda(x) =\Pr(X\notin S) = 0.$$

    This is the desired contradiction that proves the assertion.

    A Cantor variable provides the standard example of a non-discrete variable that does not have a density with respect to Lebesgue measure.

  4. "A discrete RV $X$ can have a density w.r.t. other measures such as the counting measure."

    In the widest sense this is meaningless but in the refined sense of the last phrase it is true. Literally any random variable can have a density with respect to "other measures," because every random variable determines a measure with respect to which it is absolutely continuous; namely, its cumulative distribution. Specifically, given a random variable $X,$ for every extended real number $x$ let the measure of the borel set $(-\infty, x]$ be $\Pr(X\le x).$

    However, having a density with respect to the counting measure $\nu$ essentially means $X$ has a probability distribution function $p_X:\mathbb{R}\to [0,1];$ that is, for every extended real number $x,$ $$\Pr(X\le x) = \int_{-\infty}^x p_X(x)\,\mathrm{d}\nu(x) = \sum_{y\le x} p_X(y).$$ Such a sum can make sense only when at most a countable number of the values in the sum are nonzero. Let $S = \{y\in\mathbb{R}\mid p_X(y)\gt 0\}$ be the set of all numbers with nonzero probabilities. Then $$1 = \Pr(X\le \infty) = \sum_{y\in S}p_X(y)$$ demonstrates $S$ is at most countable, QED.

  5. "The cdf $F_X$ of a discrete RV $X$ is discrete (i.e., it is not continuous)."

    This does not define discrete variables. The analysis is the same as for #1 above and the zero-inflated lognormal counterexample works just as well here.

    If "discrete ... not continuous" is intended to mean piecewise constant (in some appropriate sense), then the example of a discrete variable supported on the rational numbers in $[0,1]$ (given in the introduction) shows this is not a necessary property of discrete variables: there is no open interval in $[0,1]$ on which its cdf is constant.


Having seen the implications, it should now be apparent that 1,2,3,5 are not equivalent; that not all the statements are correct; and that the only one that could be used to define discrete random variables is a narrow interpretation of #4.

$\endgroup$
  • 1
    $\begingroup$ Thank you so much for your detailed answer. This is not the first time that you answer a questions of mine in full detail. This is very much appreciated. I havent yet understood your answer in full detail so i may come back to ask some questions regarding your answer later, but first I need to look up some concepts contained in your answer that are not yet familiar to me. $\endgroup$ – guest1 Mar 31 '20 at 9:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.