# Is there a reason other than conventions why a CDF must be defined for all real numbers

There are many cases when the sample space is not the entire set of real numbers (for instance a Bernoulli trial or sampling from an interval).

On the one hand: for the definition $F_X(x) = P(X \leq x)$, it seems fine if we define the range of $x$ to be something different than $\mathbb{R}$, for instance a finite interval (when the sample space is not equal to the set of real numbers).

On the other hand: the cumulative distribution function (CDF) is defined (has to be defined) on the entire real line.

• Is this definition on the real line convention or is there more to it?

• For what reasons is it useful, or even a necessity, that the CDF is (always) defined for the entire set of real numbers?

• Are there practical (and possibly intuitive) examples that make use of the property that the CDF is defined on the entire set of the real numbers when the sample space of an experiment is not?

• This seems like a non-problem: since one can always define the CDF of any real-valued random variable for all real numbers, perhaps the real question ought to be whether there is any value at all in constraining the domain of the CDF.
– whuber
Commented Dec 22, 2017 at 20:52
• @whuber $$\\$$Yes, it is a non-issue. There is no great value in constraining the domain of a CDF. $$\\$$ Yet, the insistence of mathematicians and statisticians on the idea that it is wrong or erroneous to do so inspires the question about the underlying reasons for this. $$\\$$ Are there fundamental mistakes luring around the corner, or is this more like a matter of definitions, conventions, simplicity, generality (but not so much a fundamental error)? Commented Dec 22, 2017 at 22:38
• I find this issue to be of no interest, because if there is any number $a$ for which you choose not to define the distribution $F$ of an RV $X$, you can always extend the domain of $F$ to include $a$ simply by defining $F(a)=\Pr(X\le a)$, as always. In brief, $F$ is always defined on $\mathbb{R}$. That's a simple mathematical fact, not a convention.
– whuber
Commented Dec 23, 2017 at 19:46
• I am not trying to argue that F does not exist on the entire $\mathbb{R}$, can not be extended to it, or that it is not the most general and complete way to describe it. $$\\$$ I am wondering whether there are technical reasons that this is also a necessary way to define it. My starting point is that for any probability function defined on an algebra on which an inequality is defined we could define a function $F(x): P(X \leq x)$. Say we express the P(someones birthday happens before day x), why are mathematicians requiring us to extend this x? When, for what utility, is this necessary? Commented Dec 23, 2017 at 21:43
• In other words. It is fine to define the PDF for the Bernouilli distribution on a restricted range e.g. : $$P(X=x) = \begin{cases} (1-p) & \text{for } k=0 \\ p & \text{for } k=1 \end{cases}$$ while this can be just as the CDF well extended to $\mathbb{R}$ $$P(X=x) = \begin{cases} (1-p) & \text{for } k=0 \\ p & \text{for } k=1 \\ 0 & \text{for } k \in \mathbb{R} \backslash \lbrace 0,1 \rbrace \end{cases}$$ the CDF has to be defined on the entire $\mathbb{R}$. Is there a technical reason for this? Commented Dec 23, 2017 at 22:09

In general it suffices to specify the CDF on the smallest interval $[x,y]$ such that $F(x)=0$ and $F(y)=1$, (note that this allows $x,y=\pm\infty$). This is equivalent to specifying $F$ just on the image $X(\Omega)$.

Otherwise, yes, it's a convention that $F(x)$ is defined on all of $(-\infty,\infty)$, specifically because $X$ is a random variable with range $\mathbb{R}$, which amounts to figuring out $P(X\leq x)$ for all $x\in\mathbb{R}$. This is an artifact of measure theory, where one is interested in sets $\{\omega: X(\omega)\leq x\}$ because they can be shown to generate the sigma algebra which makes $X$ measurable. The technical details for this are beyond the scope of this answer, and can be found in any introductory book on measure theory.

In terms of usefulness of defining $F(x)$ on a finite interval, I think actually the opposite tends to happen. As an example let $F(x)=x$ for $x\in [0,1]$ and $F(x)=0$ for $x\leq 0$, $F(x)=1$ for $x>1$. Now consider the integral $E[X]=\int_0^\infty (1-F(x))dx$, which is a way of calculating the expectation of a non-negative random variable $X$. A common mistake is to assume $F(x)=x$ for all $x$ which will give nonsense results. The correct thing to do is to truncate it to $\int_0^1(1-x)dx+\int_1^\infty (1-1)dx$. Similar issues occur a lot even with $E[X]=\int_{-\infty}^\infty xf(x)dx$, where $f(x)$ is the density of a bounded random variable.

First, let me be very clear that the distribution function $F$ of a random variable $X$ has to be defined on the entire real line $\mathbb{R}$. A very simple argument is, if you agreed with me that $$\lim_{x \to -\infty} F(x) = 0, \quad \lim_{x \to +\infty} F(x) = 1$$ holds for any distribution function $F$. If $F$ were only defined on a finite interval $[a, b]$, then how would you make sense of the above two limits?

In the subsequel, I would like to discuss your question in somewhat more theoretical flavor. There may be some applied scearnios under which people used this term loosely, which is not within the scope of my answer here. To deeply understand some arguments below, you may need some advanced probability knowledge.

First let's make it clear that the distribution function $F$ is a derivative of:

1. An original probability space $(\Omega, \mathscr{F}, P)$.
2. A random variable $X$ which is defined on $(\Omega, \mathscr{F}, P)$ and takes values on $\mathbb{R}$.

Note that all random variables, regardless of discrete, continuous, mixed, singular, are subdumed in 2. The thing that matters the concept of distribution $F$ is not whether $X$ is continuous or discrete, but it is a measurable function from $\Omega$ to $\mathbb{R}$. In other words, $X$ is a function that sends every $\omega \in \Omega$ to $X(\omega) \in \mathbb{R}$, subject to some constraint required by measurability. We can ignore the term "measurable" temporarily. The definition of the distribution function $F$ of $X$ is $$F(x) := P[X \leq x] \equiv P[\{\omega: X(\omega) \leq x\}], \quad x \in \mathbb{R}. \tag{1}$$

Well, if I stopped here, it would not answer your question at all, as $x \in \mathbb{R}$ is embedded in the definition. But let's just pause and ask ourselves why mathematicians put $\mathbb{R}$ as the domain of $F$? For example, suppose $X$ is a binomial random variable whose range is just two points $\{0, 1\}$. Is it OK that for this $X$, to restrict the domain of $F$ to be $[0, 1]$?

The usefulness of a distribution function $F$ lies on that we can directly reads from it about the probability of the event that $X$ is no more than $x$ for any $x \in \mathbb{R}$. If, you restricted the domain of $F$ on $[0, 1]$, then what's your immediate answer if someone asked you what is the probability that $X$ is no more than $100$? This may look trivial to you because you would say "come on, this is of course $1$, you even don't need a distribution function $F$ to answer it". However, it makes perfect sense for a stubborn mathematician to ask such a question and demand an answer that is only based on the information of $F$. Note that, the answer from you is based on logic, instead of relying on $F$ only and directly, i.e., you gave $1$ as the answer because you know $X$ is binomial. By contrast, probabilists introduced the concept of distribution with the hope that (and they made it) $F$ summarizes all the uncertainty information of the random variable $X$. In other words, they hoped that even you forgot what the type of $X$ at all in your head, as long as you had access to its distribution function $F$, you would still be able to answer the question "what is the probability that $X$ is no more than $100$?", by simply evaluating $F$ at $100$. It is also a piece of cake if he continued to challange you by replacing $100$ with $10000, -\pi, \sin(e), \ldots$. You might feel it is difficult to those questions if you are only willing to define $F$ on a finite interval $[0, 1]$. In summary, to completely summarize the uncertainty of a random variable $X$, its distribution has to be defined on the entire real line $\mathbb{R}$. This intuitive argument corresponds to the more technical reason that the the Borel $\sigma$-field $\mathscr{R}^1$ is generated by the collection of sets $\{(-\infty, x]: x \in \mathbb{R}\}$. If $x$ didn't go over the entire $\mathbb{R}$, the collection wouldn't be sufficient to recover $\mathscr{R}^1$ (the family of events that we are interested in).

Another reason is more mathematical: to require that the distribution function has domain $\mathbb{R}$ helps give a unified formula for computing expected value. At its original form, the expected value of $X$ is defined to be the Lebesgue integral on the original probablity space $(\Omega, \mathscr{F}, P)$ with respect to the probablity measure $P$: $$E[X] = \int_\Omega X(\omega) P(d\omega).$$

This equation is less familiar to people who didn't take measure-based probability course. They are more familiar to the formula: \begin{align} E[X] = \begin{cases} \sum_{i = 1}^m x_ip(x_i) & \text{ if } X \text{ is discrete with pmf } p, \\ \int x f(x) dx & \text{ if } X \text{ is continuous with pdf } f. \end{cases} \tag{2} \end{align}

In fact, it can be shown that by defining $F$ as in $(1)$, $E[X]$ can be expressed much more compactly than $(2)$ as: $$E[X] = \int_{\mathbb{R}} x dF(x), \tag{3}$$ regardless of $X$ is continuous or discrete. The integral in $(3)$ can be interpreted in Riemann-Stieltjes sense, which only makes sense if $F$ is defined on the entire real line. Note that $(3)$ is correct because of the equality $$\int_{\Omega} X(\omega) P(d\omega) = \int_\mathbb{R} x dF(x) \tag{4}$$ holds. Suppose, if $F$ were only defined on a finite interval of $\mathbb{R}$, then part of the integral on the right hand side of $(4)$ would be undefined and we would not be able to use the elegant formula $(3)$.

As for the example, Alex R. gives an illustrative one. If you refused to complete $F(x)$'s definition as $1$ for $x \geq 1$, it would be impossible for you to get the correct answer of $E[X]$ using the formula $E[X] = \int_0^\infty (1 - F(x)) dx$.

Let's see what is the consequence of if $F$ is not completely defined. For a $\text{Bin}(1, 1/2)$ random variable $X$, OP defines its distribution as $$F(x) = \begin{cases} 1/2 & x = 0 \\ 1 & x = 1 \end{cases} \tag{5}$$ and left $F$ undefined on other points (if I interpret correctly). The definition $(5)$ has fundamental issues as it didn't uniquely recovered the probabilistic property for $X \sim \text{Bin}(1, 1/2)$. Because OP didn't define $F$ on $(1/2, 1)$, there are infinitely many possibilities that $F$ can be on this interval, which would lead infinitely many probabilities that a $\text{Bin}(1, 1/2)$ should not have. For example, since you give me the freedom to define $F$ arbitrarily on other points other than $\{0, 1\}$ (as you gave up defining them!), I can define $F$ as $$F(x) = \begin{cases} 0 & x < 0 \\ 1/2 & 0 \leq x < 1/2 \\ 3/4 & 1/2 \leq x < 1 \\ 1 & x \geq 1 \end{cases} \tag{6}$$ Note my definition of $F$ agrees with yours on $\{0, 1\}$ (I could have made it more bizzare). However, $(6)$ leads to $$P(X = 1/2) = P(X \leq 1/2) - P(X < 1/2) = F(1/2) - F(1/2-) = 3/4 - 1/2 = 1/4.$$ Is it possible for that to happen for $X \sim \text{Bin}(1, 1/2)$?

The culprit of this is simply because you didn't define $F$ unambiguously on the entire real line $\mathbb{R}$. Again, to define $F$ completely is not only for elegance, it is more about unambiguity and accuracy to describe the probabilistic property of a random variable. The ambiguity above can be eliminated if you define $F$ conventionally as follows: $$F(x) = \begin{cases} 0 & x < 0 \\ 1/2 & 0 \leq x < 1 \\ 1 & x \geq 1 \end{cases} \tag{7}$$

Based on $(7)$, you get all correct probabilities of a Bernoulli r.v. should have.

You might argue that $X$ can't take value at $1/2$ on the above example, but remember that my point is, the distribution function $F$ should be such that it gives all correct probabilities by itself alone of a random variable, without any other further prior information. In other words, $X$ and $F$ should be able to determine each other unambiguously, given the availability of one of them. Can your representation $(5)$ unambiguously identify a $\text{Bin}(1, 1/2)$ r.v. successfully? I showed you it cannot.

Therefore, elegance is second place, correctness is the probabilist's first concern.

• Nice, long, story. I could extract three points. 1) The binary relation $\leq$ makes it possible to extend the range to R and relates to the logic $P(X\leq x)\geq P(X\leq y)$ if $x\geq y$. 2) The definition of F as a differentiable function on $\mathbb{R}$ makes it possible to create generalizable expressions for expectation values (I have to look up what Riemann-Stieltjes was again, but is the CDF for discrete variables working well in this way?) 3) It prevents misinterpretation when performing calculations like $\int_0^\infty (1-F(x))dx$ if F(x) isnt properly defined on $[0,\infty]$. Commented Dec 22, 2017 at 22:31
• So it is not yet really creating major issues is it? Only when we wish to express integrals in certain elegant ways. Commented Dec 22, 2017 at 22:34
• @MartijnWeterings Basically your interpretation is correct. Some corrections and remarks to your comment. $F$ does not need to be a differentiable function on $\mathbb{R}$, in fact, it could be very irregular. For R-S integral representation for discrete r.v.s, definitely, if you reviewed the R-S integral, you would find (3) worked perfectly to derive the first formula in (2). Commented Dec 22, 2017 at 22:41
• @MartijnWeterings Considering "major issues", it really depends on users. For me (who think statistics has to be built on a solid, rigorous probability foundation), it's a very big issue if you don't allow me to define $F$ on the entire real line :) (for reasons that I gave in my answer). But for more applied-statistics inclined people, well, it indeed is not a big issue as anyway how $F$ is defined has nothing to do with $p$-values or prediction errors. Commented Dec 22, 2017 at 22:45
• - I can follow the reason of the application of the R-S integration now. $\qquad$ - For that case F need not be a differentiable function, but only a monotone real valued function. $\qquad$ - It is still not so clear why it needs to be the entire real line, except that it allows you to conveniently integrate to infinity. Commented Dec 22, 2017 at 22:52