3
$\begingroup$

At several sources I have encountered the following two definitions of a continuous random variable associated with uncountable sets:

a) uncountable range: The random variable X is continuous if its range is uncountable infinite/set of possible values is uncountable infinite.

b) uncountable sample space: The random variable X is continuous if the sample space is uncountable infinite.

I have already learned that they are wrong but dont understand why. Hence, my questions would be how they relate to each other and in particular why they are wrong definitions?

$\endgroup$
3
$\begingroup$

Well, even if the range (or support set) of the random variable $X$ is uncountable, $X$ do not necessarily have a density. The answer by @Sebastian mentions measure, and specifically counting measure. But counting measure on an uncountable set isn't very useful, for instance, it is not $\sigma$-finite. So not very useful in probability. There is an interesting counterexample, the Cantor distribution have support on an uncountable set---the Cantor (middle-third) set, but do not have a density, so is not continuous. Neither is it discrete, it is singular. See How to sample from Cantor distribution?, Is probability theory the study of non-negative functions that integrate/sum to one? and search ...

Such singular distributions are not common in statistics (except as counterexample), but are ubiquitous in other areas. See https://mathoverflow.net/questions/163325/singular-distributions-applications-and-instances. A case in point is dynamics, with the famous Smale's horseshoe, where distributions supported on dynamical Cantor sets abound.

$\endgroup$
  • $\begingroup$ Thank you for the hint w.r.t. the counting measure. So Sebastian mentioned that discrete RVs can have a density w.r.t. to the counting measure, but if the counting measure is not useful in probability, i.e., if I dont consider the counting measure, is it then correct to say that a discrete RV never has a density? $\endgroup$ – guest1 Mar 25 at 18:22
  • $\begingroup$ No, a discrete random variable (that is, $X$ such that there is a countable set with probability 1) has a density with respect to counting measure (on that countable set). But counting measures on uncountable sets (like the Cantor set) are not very useful <in probability. $\endgroup$ – kjetil b halvorsen Mar 25 at 19:05
  • $\begingroup$ Okay, so maybe I am a bit confused now. a) Is the correct definition of a discrete RV that there is a countable subset of the sample space that takes probability 1? b) I thought the definition of a continuous RV is that it has a density. But if a discrete RV also has a density then this definition doesnt make sense. Is the correct way to say that a continuous RV X is one that has a density w.r.t Lebesgue measure? I.e., does a discrete RV not have a density w.r.t Lebesgue measure? $\endgroup$ – guest1 Mar 26 at 7:28
  • $\begingroup$ And another question to your naming convention: By range you mean the image of the random variable (and not the domain)? But isnt the support a synonym to domain? $\endgroup$ – guest1 Mar 26 at 16:20
3
$\begingroup$

The problem with both characterizations is that they ignore the underlying probabilities.

Recall that a random variable $X$ is a function that assigns real numbers to elements of the sample space. If a considerable part of the domain of $X$ has no probability, then the range of $X$ may have virtually any property whatsoever but that won't tell you a thing about the distribution of $X.$

Here are the mathematical details.

By definition, a random variable $X$ has a distribution function defined by $$F_X(x)=\Pr(X\le x)$$ for all numbers $x.$ $X$ is continuous if and only if $F_X$ is a continuous function everywhere.

As a counterexample to both (a) and (b), let $\Omega=[0,1]$ be the sample space of all real numbers between $0$ and $1$ inclusive with its usual Borel sigma-algebra. $\Omega$ is uncountable. Let $\mathbb P$ be the normalized counting measure on $\{0,1\}.$ This means the value of $\mathbb P$ on any event $\mathcal E\subset \Omega$ is the sum of two values: $0$ if $0\notin \mathcal E$ or $1/2$ if $0\in\mathcal E;$ plus $0$ if $1\notin \mathcal E$ or $1/2$ if $1\in\mathcal E.$ This is a standard way to model the flip of a fair coin, for instance.

Define a random variable by $$X:\Omega\to\mathbb{R},\quad X(\omega)=\omega.$$ By one standard definition, the range of $X$ is the smallest interval $[a,b]\subset\mathbb R$ for which $\mathbb{P}(X\in[a,b])=1.$ Clearly $0\in[a,b],$ $1\in[a,b],$ and $\mathbb{P}([0,1])=1,$ whence the range of $X$ is $[0,1].$

(Notice how this models the intuition in the introductory paragraphs: although $X$ takes on uncountably many possible values, the only values that have any nonzero probability are limited to just the finite set $\{0,1\}.$)

Although the range of $X$ is the uncountable set $[0,1],$ the distribution function $F_X$ is piecewise constant, jumping from $0$ to $1/2$ at $x=0$ and from $1/2$ to $1$ at $x=1.$ (This is the Bernoulli$(1/2)$ CDF.) $F_X$ is obviously not continuous at either point, even though (a) the range of $X$ is uncountable and (b) the sample space $\Omega$ is uncountable.

$\endgroup$
  • $\begingroup$ Thank you for your answer. I have two questions referring to it: 1.) you are mentioning the counting measure just like Sebastian in his answer. I am not really familiar with that measure: Isnt the Lebesgue measure the only one important for probability ? 2.) Are you talking about continuous RVs? If so, would the matters change if we would be talking about absolutely continuous RVs? $\endgroup$ – guest1 Mar 30 at 14:53
  • $\begingroup$ @guest1 The counting measure is ubiquitous in probability: it's the one to which students are introduced. For further clarification of these issues, please see my answer to your question. $\endgroup$ – whuber Mar 30 at 15:11
2
$\begingroup$

Consider the example where your sample space $\Omega$ = $\mathbb{R}$. This is uncountably infinite. However whether the RV is continuous depends on the used measure. If you would use $\mu = \#$ (i.e. the counting-measure) you could still easily defined a density with respect to $\mu$ that would induce a discrete distribution.

In general whether a distribution is discrete or continuous depends on the distribution function. It can of course also be a mixture of both (or to make things even weirded be 'singular', see e.g. the Cantor-distribution).

$\endgroup$
  • $\begingroup$ Hi, thank you for your answer. I think you mean that the real numbers are uncountable infinite, right? Another question: How can a discrete random variable have a density (pdf). I though only continuous random variables have pdfs and discrete random variables have pmfs? $\endgroup$ – guest1 Mar 25 at 10:42
  • $\begingroup$ Ah yes this was a typo. In an abstract measure-theoretic sense pmfs can also be defined via densities with respect to the counting measure en.wikipedia.org/wiki/Radon%E2%80%93Nikodym_theorem. If you are unfamiliar with measure theory just ignore that word and think of it as a pmf. The main idea is that the sample space can be uncountably infinite but there could still be a .5 mass on 1 and a .5 mass on 2, implying a discrete distribution $\endgroup$ – Sebastian Mar 25 at 10:45
  • $\begingroup$ So it would mean that a discrete random variable can have an uncountably infinite sample space but only a countably finite number of points are allowed to have a probability unequal to zero? $\endgroup$ – guest1 Mar 25 at 11:04
  • $\begingroup$ Exactly, that's what I ment to say. $\endgroup$ – Sebastian Mar 25 at 11:08
  • $\begingroup$ Okay but how does this contrast to a continuous RV X? Because my above definition of a discrete RV X implies that there is also an uncountable number of points in the sample space with probability zero. But this is exactly the definition/characterization of a continuous random variable, no? A continuous RV X is one that has Pr(X=x)=0 for all x, i.e., for an uncountable number of points in the sample space $\endgroup$ – guest1 Mar 25 at 11:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.