5
$\begingroup$

At several sources I have encountered the following two definitions of a continuous random variable associated with uncountable sets:

a) uncountable range: The random variable X is continuous if its range is uncountable infinite/set of possible values is uncountable infinite.

b) uncountable sample space: The random variable X is continuous if the sample space is uncountable infinite.

I have already learned that they are wrong but dont understand why. Hence, my questions would be how they relate to each other and in particular why they are wrong definitions?

$\endgroup$

3 Answers 3

7
$\begingroup$

The problem with both characterizations is that they ignore the underlying probabilities.

Recall that a random variable $X$ is a function that assigns real numbers to elements of the sample space. If a considerable part of the domain of $X$ has no probability, then the range of $X$ may have virtually any property whatsoever but that won't tell you a thing about the distribution of $X.$

Here are the mathematical details.

By definition, a random variable $X$ has a distribution function defined by $$F_X(x)=\Pr(X\le x)$$ for all numbers $x.$ $X$ is continuous if and only if $F_X$ is a continuous function everywhere.

As a counterexample to both (a) and (b), let $\Omega=[0,1]$ be the sample space of all real numbers between $0$ and $1$ inclusive with its usual Borel sigma-algebra. $\Omega$ is uncountable. Let $\mathbb P$ be the normalized counting measure on $\{0,1\}.$ This means the value of $\mathbb P$ on any event $\mathcal E\subset \Omega$ is the sum of two values: $0$ if $0\notin \mathcal E$ or $1/2$ if $0\in\mathcal E;$ plus $0$ if $1\notin \mathcal E$ or $1/2$ if $1\in\mathcal E.$ This is a standard way to model the flip of a fair coin, for instance.

Define a random variable by $$X:\Omega\to\mathbb{R},\quad X(\omega)=\omega.$$ By one standard definition, the range of $X$ is the smallest interval $[a,b]\subset\mathbb R$ for which $\mathbb{P}(X\in[a,b])=1.$ Clearly $0\in[a,b],$ $1\in[a,b],$ and $\mathbb{P}([0,1])=1,$ whence the range of $X$ is $[0,1].$

(Notice how this models the intuition in the introductory paragraphs: although $X$ takes on uncountably many possible values, the only values that have any nonzero probability are limited to just the finite set $\{0,1\}.$)

Although the range of $X$ is the uncountable set $[0,1],$ the distribution function $F_X$ is piecewise constant, jumping from $0$ to $1/2$ at $x=0$ and from $1/2$ to $1$ at $x=1.$ (This is the Bernoulli$(1/2)$ CDF.) $F_X$ is obviously not continuous at either point, even though (a) the range of $X$ is uncountable and (b) the sample space $\Omega$ is uncountable.

$\endgroup$
4
  • $\begingroup$ Thank you for your answer. I have two questions referring to it: 1.) you are mentioning the counting measure just like Sebastian in his answer. I am not really familiar with that measure: Isnt the Lebesgue measure the only one important for probability ? 2.) Are you talking about continuous RVs? If so, would the matters change if we would be talking about absolutely continuous RVs? $\endgroup$
    – guest1
    Mar 30, 2020 at 14:53
  • 1
    $\begingroup$ @guest1 The counting measure is ubiquitous in probability: it's the one to which students are introduced. For further clarification of these issues, please see my answer to your question. $\endgroup$
    – whuber
    Mar 30, 2020 at 15:11
  • $\begingroup$ Hey, by continuous everywhere, do you mean cont. on entire reals or just wherever it's defined? I see everywhere that CRV is defined as one with uncountable range so I'm trying to understand the difference between that and your definition. When you say $F(x)$ needs to be continuous, how does it capture the idea that the RV is continuous? In what way is it cont.? I understand that if $F(x)$ is cont. then clearly the PDF$f(x)$needs to be integrable, not necessarily cont. but I can't seem to make the connection as to what does that have anything to do with RV being cont.? Any help is appreciated! $\endgroup$
    – William
    Oct 7, 2023 at 7:10
  • $\begingroup$ @William By definition, all distribution functions are defined everywhere on the reals: there's no restriction on what $x$ might be. For further discussion, please research our posts on continuous and absolutely continuous distributions, such as stats.stackexchange.com/a/298434/919, which explicitly answers your other questions. $\endgroup$
    – whuber
    Oct 7, 2023 at 11:02
4
$\begingroup$

Consider the example where your sample space $\Omega$ = $\mathbb{R}$. This is uncountably infinite. However whether the RV is continuous depends on the used measure. If you would use $\mu = \#$ (i.e. the counting-measure) you could still easily defined a density with respect to $\mu$ that would induce a discrete distribution.

In general whether a distribution is discrete or continuous depends on the distribution function. It can of course also be a mixture of both (or to make things even weirded be 'singular', see e.g. the Cantor-distribution).

$\endgroup$
12
  • $\begingroup$ Hi, thank you for your answer. I think you mean that the real numbers are uncountable infinite, right? Another question: How can a discrete random variable have a density (pdf). I though only continuous random variables have pdfs and discrete random variables have pmfs? $\endgroup$
    – guest1
    Mar 25, 2020 at 10:42
  • 1
    $\begingroup$ Ah yes this was a typo. In an abstract measure-theoretic sense pmfs can also be defined via densities with respect to the counting measure en.wikipedia.org/wiki/Radon%E2%80%93Nikodym_theorem. If you are unfamiliar with measure theory just ignore that word and think of it as a pmf. The main idea is that the sample space can be uncountably infinite but there could still be a .5 mass on 1 and a .5 mass on 2, implying a discrete distribution $\endgroup$
    – Sebastian
    Mar 25, 2020 at 10:45
  • $\begingroup$ So it would mean that a discrete random variable can have an uncountably infinite sample space but only a countably finite number of points are allowed to have a probability unequal to zero? $\endgroup$
    – guest1
    Mar 25, 2020 at 11:04
  • $\begingroup$ Exactly, that's what I ment to say. $\endgroup$
    – Sebastian
    Mar 25, 2020 at 11:08
  • $\begingroup$ Okay but how does this contrast to a continuous RV X? Because my above definition of a discrete RV X implies that there is also an uncountable number of points in the sample space with probability zero. But this is exactly the definition/characterization of a continuous random variable, no? A continuous RV X is one that has Pr(X=x)=0 for all x, i.e., for an uncountable number of points in the sample space $\endgroup$
    – guest1
    Mar 25, 2020 at 11:17
4
$\begingroup$

Well, even if the range (or support set) of the random variable $X$ is uncountable, $X$ do not necessarily have a density. The answer by @Sebastian mentions measure, and specifically counting measure. But counting measure on an uncountable set isn't very useful, for instance, it is not $\sigma$-finite. So not very useful in probability.

There is an interesting counterexample, the Cantor distribution have support on an uncountable set --- the Cantor (middle-third) set, but do not have a density, so is not absolutely continuous. Neither is it discrete, it is singular. See How to sample from Cantor distribution?, Is probability theory the study of non-negative functions that integrate/sum to one? and search ...

Such singular distributions are not common in statistics (except as counterexample), but are ubiquitous in other areas. See singular distributions applications and instances. A case in point is dynamics, with the famous Smale's horseshoe, where distributions supported on dynamical Cantor sets abound.

$\endgroup$
5
  • $\begingroup$ Thank you for the hint w.r.t. the counting measure. So Sebastian mentioned that discrete RVs can have a density w.r.t. to the counting measure, but if the counting measure is not useful in probability, i.e., if I dont consider the counting measure, is it then correct to say that a discrete RV never has a density? $\endgroup$
    – guest1
    Mar 25, 2020 at 18:22
  • 1
    $\begingroup$ No, a discrete random variable (that is, $X$ such that there is a countable set with probability 1) has a density with respect to counting measure (on that countable set). But counting measures on uncountable sets (like the Cantor set) are not very useful <in probability. $\endgroup$ Mar 25, 2020 at 19:05
  • $\begingroup$ Okay, so maybe I am a bit confused now. a) Is the correct definition of a discrete RV that there is a countable subset of the sample space that takes probability 1? b) I thought the definition of a continuous RV is that it has a density. But if a discrete RV also has a density then this definition doesnt make sense. Is the correct way to say that a continuous RV X is one that has a density w.r.t Lebesgue measure? I.e., does a discrete RV not have a density w.r.t Lebesgue measure? $\endgroup$
    – guest1
    Mar 26, 2020 at 7:28
  • $\begingroup$ And another question to your naming convention: By range you mean the image of the random variable (and not the domain)? But isnt the support a synonym to domain? $\endgroup$
    – guest1
    Mar 26, 2020 at 16:20
  • 2
    $\begingroup$ A continuous random variable has a cumulative distribution function that is continuous. Not to be confused with an absolutely continuous random variable, which enjoys a density wrt the Lebesgue measure on $\mathbb R$. $\endgroup$
    – Xi'an
    Dec 21, 2020 at 8:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.