73

Every probability distribution on (a subset of) $\mathbb R^n$ has a cumulative distribution function, and it uniquely defines the distribution. So, in this sense, the CDF is indeed as fundamental as the distribution itself. A probability density function, however, exists only for (absolutely) continuous probability distributions. The simplest example of a ...


63

All this may sound complicated at first, but it is essentially about something very simple. By cumulative distribution function we denote the function that returns probabilities of $X$ being smaller than or equal to some value $x$, $$ \Pr(X \le x) = F(x).$$ This function takes as input $x$ and returns values from the $[0, 1]$ interval (probabilities)&...


49

The Kolmogorov-Smirnov test assesses the hypothesis that a random sample (of numerical data) came from a continuous distribution that was completely specified without referring to the data. Here is the graph of the cumulative distribution function (CDF) of such a distribution. A sample can be fully described by its empirical (cumulative) distribution ...


46

Notation, upper and lower case https://en.wikipedia.org/wiki/Notation_in_probability_and_statistics Random variables are usually written in upper case roman letters: $X$, $Y$, etc. Particular realizations of a random variable are written in corresponding lower case letters. For example $x_1$, $x_2$, …, $x_n$ could be a sample corresponding to the random ...


42

?density points out that it uses approx to do linear interpolation already; ?approx points out that approxfun generates a suitable function: x <- log(rgamma(150,5)) df <- approxfun(density(x)) plot(density(x)) xnew <- c(0.45,1.84,2.3) points(xnew,df(xnew),col=2) By use of integrate starting from an appropriate distance below the minimum in the ...


39

Convolution calculations associated with distributions of random variables are all mathematical manifestations of the Law of Total Probability. In the language of my post at What is meant by a “random variable”?, A pair of random variables $(X,Y)$ consists of a box of tickets on each of which are written two numbers, one designated $X$ and the other $Y$. ...


36

Your confusion seems to arise from conflating random variables with their distributions. To "unlearn" this confusion, it might help to take a couple of steps back, empty your mind for a moment, forget about any fancy formalisms like probability spaces and sigma-algebras (if it helps, pretend you're back in elementary school and have never heard of any of ...


30

Let $X$ be a random variable. The cumulative distribution function $F(x)$ gives the $P(X \leq x)$. An empirical cumulative distribution function function $G(x)$ gives $P(X \leq x)$ based on the observations in your sample. The distinction is which probability measure is used. For the empirical CDF, you use the probability measure defined by the frequency ...


28

Assume $F_X$ is continuous and increasing. Define $Z = F_X(X)$ and note that $Z$ takes values in $[0, 1]$. Then $$F_Z(x) = P(F_X(X) \leq x) = P(X \leq F_X^{-1}(x)) = F_X(F_X^{-1}(x)) = x.$$ On the other hand, if $U$ is a uniform random variable that takes values in $[0, 1]$, $$F_U(x) = \int_R f_U(u)\,du =\int_0^x \,du =x.$$ Thus $F_Z(x) = F_U(x)$ for ...


25

Where a distinction is made between probability function and density*, the pmf applies only to discrete random variables, while the pdf applies to continuous random variables. * formal approaches can encompass both and use a single term for them The cdf applies to any random variables, including ones that have neither a pdf nor pmf. (A mixed distribution ...


25

Because this comes up often in some systems (for instance, Mathematica insists on expressing the Normal CDF in terms of $\text{Erf}$), it's good to have a thread like this that documents the relationship. By definition, the Error Function is $$\text{Erf}(x) = \frac{2}{\sqrt{\pi}}\int_0^x e^{-t^2} \mathrm{d}t.$$ Writing $t^2 = z^2/2$ implies $t = z / \sqrt{...


24

No, this is false. That point is the median, and it is not equal to the mean in all cases, for example, an exponential distribution. $$ X\sim \exp(1)\\ \mathbb E[X]=1\\ \operatorname{median}(X)=\log 2\approx 0.69 $$ We can simulate this in software, such as R. set.seed(2021) X <- rexp(10000, 1) mean(X) # approximately 1 median(X) # approximately 0.69 ...


20

I don't see any sense in not "believing" the Q-Q plot (if you've produced it properly); it's just a graphical representation of the reality of your data, juxtaposed with the definitional distribution. Clearly it's not a perfect match, but if it's good enough for your purposes, that may be more or less the end of the story. You may want to check out this ...


18

There's no need to integrate anything if you know the cdf of the kernel itself. I believe this is straightforward for all the common kernels. Note that a KDE is a mixture density the cdf of a mixture is the mixture of the cdfs. that is, if $\hat{f}(x)=\frac{1}{n}\sum_i f_i(x)$ is your KDE at $x$, then $\hat{F}(x)=\frac{1}{n}\sum_i F_i(x)$. Take a ...


18

The absolute value of this area is $$\int_{x=-\infty}^\infty \lvert F(x) - G(x)\rvert \,\mathrm{d}x,$$ which note – at least for continuous distributions – is exactly equal to $$\int_{x=-\infty}^\infty \lvert F^{-1}(x) - G^{-1}(x)\rvert \,\mathrm{d}x.$$ In one dimension, the latter is the 1-Wasserstein distance, the 1-Kantorovich distance, or the "earth-...


18

Problems like this, where you want to differentiate the product of a bunch of functions that depend on your variable of interest, can be dealt with by logarithmic differentiation. Let $\Phi$ and $\phi$ denote the CDF and PDF of the standard normal distribution (respectively). Since the normal random variables in your question have the same variance you get:...


17

What you could do is create multiple samples from your theoretical distribution and plot those on the background of your QQ-plot. That will give you an idea of what kind of variability you can reasonably expect from just sampling. You can extend that idea to create an envelope around the theoretical line, using the example from pages 86-89 of : Venables, ...


17

The proper terminology is Cumulative Distribution Function, (CDF). The CDF is defined as $$F_X(x) = \mathrm{P}\{X \leq x\}.$$ With this definition, the nature of the random variable $X$ is irrelevant: continuous, discrete, or hybrids all have the same definition. As you note, for a discrete random variable the CDF has a very different appearance than for a ...


17

I believe your econometrics professor was thinking something along the following lines. Consider the function $F$ with domiain $[0, 1]$ defined by $$F(x) = \frac{1}{2}x \ \text{for} \ x < \frac{1}{2} $$ $$F(x) = \frac{1}{2}x + \frac{1}{2} \ \text{for} \ x \geq \frac{1}{2} $$ This is a discontinuous function, but a completely valid CDF for some ...


17

The inverse cdf method operates even when the cdf is not invertible, using the generalised inverse$$F^-(u)=\sup\{x;\ F(x)\le u\}\tag{1}$$ which is always defined for $u\in(0,1)$. When there is no solution in $x$ to the equation $$F(x)=u$$ it means that $F$ has a jump between a value less than $u$, $u-\epsilon$ and a value more than $u$, $u+\eta$. Hence the ...


16

The empirical distribution function $\hat{F}(\cdot)$ is a step function by construction, since it puts a probability (Dirac) mass of $1/n$ on every term in the sample, $(x_1,\ldots,x_n)$, hence jumps by the same factor $1/n$ from one observation to the next. As a result, it is not everywhere differentiable and cannot be associated with a probability density ...


15

Simulation from a truncated normal is easily done if you have access to a proper normal quantile function. For instance, in R, simulating $$ \mathcal{N}_a^b(\mu,\sigma^2)$$where $a$ and $b$ denote the lower and upper bounds can be done by inverting the cdf $$\dfrac{\Phi(\sigma^{-1}\{x-\mu\})-\Phi(\sigma^{-1}\{a-\mu\})}{\Phi(\sigma^{-1}\{b-\mu\})-\Phi(\sigma^{...


15

Is there any difference between Empirical CDF and CDF? Yes, they're different. An empirical cdf is a proper cdf, but empirical cdfs will always be discrete even when not drawn from a discrete distribution, while the cdf of a distribution can be other things besides discrete. If you treat a sample as if it were a population of values, each one equally ...


14

Let us recall some things. Let $(\Omega,A,P)$ be a probability space, $\Omega$ is our sample set, $A$ is our $\sigma$-algebra, and $P$ is a probability function defined on $A$. A random variable is a measurable function $X:\Omega \to \mathbb{R}$ i.e. $X^{-1}(S) \in A$ for any Lebesgue measurable subset in $\mathbb{R}$. If you are not familiar with this ...


14

It's a common trick. If $X = \min(Y_1,Y_2)$ and $F$, $F_X$ are the CDFs of the $Y_i$s and $X$, respectively, then \begin{align*} F_X(x) &= 1 - P(X > x) \\ &= 1- P(Y_1 > x, Y_2 > x) \\ &= 1 - P(Y_1 > x)P(Y_2 > x) \text{ independence}\\ &= 1 - [1-F(x)][1-F(x)] \text{ identicalness}. \end{align*}


13

As noted by Wikipedia, probability distribution function is ambiguous term: A probability distribution function is some function that may be used to define a particular probability distribution. Depending upon which text is consulted, the term may refer to: a cumulative distribution function, a probability mass function, and/or a ...


13

A transform of a random variable $X$ by a measurable function $T:\mathcal{X}\longrightarrow\mathcal{Y}$ is another random variable $Y=T(X)$ which distribution is given by the inverse probability transform $$\mathbb{P}(Y\in A) = \mathbb{P}(X\in\{x;\,T(x)\in A\})\stackrel{\text{def}}{=} \mathbb{P}(X\in T^{-1}(A))$$ for all sets $A$ such that $\{x;\,T(x)\in A\}$...


13

Because if $x \leq y$, then if $X \leq x$, it follows that $X \leq y$. Therefore, $P(X \leq x) \leq P(X \leq y)$. More generally, probabilities are monotone in the sense that if $A$ and $B$ are events and $A \subseteq B$, then $P(A) \leq P(B)$. This follows from writing $B$ as the disjoint union of $A$ and $B \setminus A$, whence by the probability axioms $P(...


13

An enormous amount of data is needed to accurately estimate a distribution nonparametrically, especially a continuous one. Even then, some assumptions about the smoothness of the distribution are needed for filling the gaps (interpolating) between the observed values and other assumptions are needed for extrapolating outside the observed data range. With a ...


13

The mean is the point where the area left and right between the CDF and the vertical line through the mean are equal. For symmetric distributions, the left and the right side/area have the same shape and you get that the mean is at the point where the CDF is equal to 0.5 (also the median). For non-symmetric distributions the shapes are different, so this ...


Only top voted, non community-wiki answers of a minimum length are eligible