Hot answers tagged

69

Every probability distribution on (a subset of) $\mathbb R^n$ has a cumulative distribution function, and it uniquely defines the distribution. So, in this sense, the CDF is indeed as fundamental as the distribution itself. A probability density function, however, exists only for (absolutely) continuous probability distributions. The simplest example of a ...


42

Notation, upper and lower case https://en.wikipedia.org/wiki/Notation_in_probability_and_statistics Random variables are usually written in upper case roman letters: $X$, $Y$, etc. Particular realizations of a random variable are written in corresponding lower case letters. For example $x_1$, $x_2$, …, $x_n$ could be a sample corresponding to the ...


40

All this may sound complicated at first, but it is essentially about something very simple. By cumulative distribution function we denote the function that returns probabilities of $X$ being smaller than or equal to some value $x$, $$ \Pr(X \le x) = F(x).$$ This function takes as input $x$ and returns values from the $[0, 1]$ interval (probabilities)&...


36

?density points out that it uses approx to do linear interpolation already; ?approx points out that approxfun generates a suitable function: x <- log(rgamma(150,5)) df <- approxfun(density(x)) plot(density(x)) xnew <- c(0.45,1.84,2.3) points(xnew,df(xnew),col=2) By use of integrate starting from an appropriate distance below the minimum in the ...


32

The ecdf function applied to a data sample returns a function representing the empirical cumulative distribution function. For example: > X = rnorm(100) # X is a sample of 100 normally distributed random variables > P = ecdf(X) # P is a function giving the empirical CDF of X > P(0.0) # This returns the empirical CDF at zero (should be ...


28

Your confusion seems to arise from conflating random variables with their distributions. To "unlearn" this confusion, it might help to take a couple of steps back, empty your mind for a moment, forget about any fancy formalisms like probability spaces and sigma-algebras (if it helps, pretend you're back in elementary school and have never heard of any of ...


27

Let $X$ be a random variable. The cumulative distribution function $F(x)$ gives the $P(X \leq x)$. An empirical cumulative distribution function function $G(x)$ gives $P(X \leq x)$ based on the observations in your sample. The distinction is which probability measure is used. For the empirical CDF, you use the probability measure defined by the frequency ...


25

Where a distinction is made between probability function and density*, the pmf applies only to discrete random variables, while the pdf applies to continuous random variables. * formal approaches can encompass both and use a single term for them The cdf applies to any random variables, including ones that have neither a pdf nor pmf. (A mixed distribution ...


20

Assume $F_X$ is continuous and increasing. Define $Z = F_X(X)$ and note that $Z$ takes values in $[0, 1]$. Then $$F_Z(x) = P(F_X(X) \leq x) = P(X \leq F_X^{-1}(x)) = F_X(F_X^{-1}(x)) = x.$$ On the other hand, if $U$ is a uniform random variable that takes values in $[0, 1]$, $$F_U(x) = \int_R f_U(u)\,du =\int_0^x \,du =x.$$ Thus $F_Z(x) = F_U(x)$ for ...


19

Convolution calculations associated with distributions of random variables are all mathematical manifestations of the Law of Total Probability. In the language of my post at What is meant by a “random variable”?, A pair of random variables $(X,Y)$ consists of a box of tickets on each of which are written two numbers, one designated $X$ and the other $...


18

I don't see any sense in not "believing" the Q-Q plot (if you've produced it properly); it's just a graphical representation of the reality of your data, juxtaposed with the definitional distribution. Clearly it's not a perfect match, but if it's good enough for your purposes, that may be more or less the end of the story. You may want to check out this ...


17

Because this comes up often in some systems (for instance, Mathematica insists on expressing the Normal CDF in terms of $\text{Erf}$), it's good to have a thread like this that documents the relationship. By definition, the Error Function is $$\text{Erf}(x) = \frac{2}{\sqrt{\pi}}\int_0^x e^{-t^2} \mathrm{d}t.$$ Writing $t^2 = z^2/2$ implies $t = z / \sqrt{...


16

What you could do is create multiple samples from your theoretical distribution and plot those on the background of your QQ-plot. That will give you an idea of what kind of variability you can reasonably expect from just sampling. You can extend that idea to create an envelope around the theoretical line, using the example from pages 86-89 of : Venables, ...


16

The proper terminology is Cumulative Distribution Function, (CDF). The CDF is defined as $$F_X(x) = \mathrm{P}\{X \leq x\}.$$ With this definition, the nature of the random variable $X$ is irrelevant: continuous, discrete, or hybrids all have the same definition. As you note, for a discrete random variable the CDF has a very different appearance than for a ...


16

I believe your econometrics professor was thinking something along the following lines. Consider the function $F$ with domiain $[0, 1]$ defined by $$F(x) = \frac{1}{2}x \ \text{for} \ x < \frac{1}{2} $$ $$F(x) = \frac{1}{2}x + \frac{1}{2} \ \text{for} \ x \geq \frac{1}{2} $$ This is a discontinuous function, but a completely valid CDF for some ...


16

The absolute value of this area, i.e. $$\int_{x=-\infty}^\infty \lvert F(x) - G(x)\rvert \,\mathrm{d}x,$$ is known as the 1-Wasserstein distance, the 1-Kantorovich distance, or the "earth-mover's distance." It is quite a reasonable distance between probability distributions, which is easy to compute between one-dimensional distributions based on their cdfs. ...


13

Let us recall some things. Let $(\Omega,A,P)$ be a probability space, $\Omega$ is our sample set, $A$ is our $\sigma$-algebra, and $P$ is a probability function defined on $A$. A random variable is a measurable function $X:\Omega \to \mathbb{R}$ i.e. $X^{-1}(S) \in A$ for any Lebesgue measurable subset in $\mathbb{R}$. If you are not familiar with this ...


13

Simulation from a truncated normal is easily done if you have access to a proper normal quantile function. For instance, in R, simulating $$ \mathcal{N}_a^b(\mu,\sigma^2)$$where $a$ and $b$ denote the lower and upper bounds can be done by inverting the cdf $$\dfrac{\Phi(\sigma^{-1}\{x-\mu\})-\Phi(\sigma^{-1}\{a-\mu\})}{\Phi(\sigma^{-1}\{b-\mu\})-\Phi(\sigma^{...


13

It's a common trick. If $X = \min(Y_1,Y_2)$ and $F$, $F_X$ are the CDFs of the $Y_i$s and $X$, respectively, then \begin{align*} F_X(x) &= 1 - P(X > x) \\ &= 1- P(Y_1 > x, Y_2 > x) \\ &= 1 - P(Y_1 > x)P(Y_2 > x) \text{ independence}\\ &= 1 - [1-F(x)][1-F(x)] \text{ identicalness}. \end{align*}


12

Is there any difference between Empirical CDF and CDF? Yes, they're different. An empirical cdf is a proper cdf, but empirical cdfs will always be discrete even when not drawn from a discrete distribution, while the cdf of a distribution can be other things besides discrete. If you treat a sample as if it were a population of values, each one equally ...


12

There's no need to integrate anything if you know the cdf of the kernel itself. I believe this is straightforward for all the common kernels. Note that a KDE is a mixture density the cdf of a mixture is the mixture of the cdfs. that is, if $\hat{f}(x)=\frac{1}{n}\sum_i f_i(x)$ is your KDE at $x$, then $\hat{F}(x)=\frac{1}{n}\sum_i F_i(x)$. Take a ...


11

The cumulative distribution function of a random variable $X$ has nothing to do with summing the random variable. It is the probability that $X$ will take a value less than or equal to $x$. And of course, the probability that a value randomly sampled from your vector $(1, \dots, 1000)$ is less than or equal to 200 is exactly half the probability that it ...


11

The empirical CDF is built from an actual data set (in the plot below, I used 100 samples from a standard normal distribution). The CDF is a theoretical construct - it is what you would see if you could take infinitely many samples. The empirical CDF usually approximates the CDF quite well, especially for large samples (in fact, there are theorems about how ...


11

A transform of a random variable $X$ by a measurable function $T:\mathcal{X}\longrightarrow\mathcal{Y}$ is another random variable $Y=T(X)$ which distribution is given by the inverse probability transform $$\mathbb{P}(Y\in A) = \mathbb{P}(X\in\{x;\,T(x)\in A\})\stackrel{\text{def}}{=} \mathbb{P}(X\in T^{-1}(A))$$ for all sets $A$ such that $\{x;\,T(x)\in A\}$...


11

Why is it starting from the top-left corner? The standard* definition of a CDF is $$ F_X(x) := \mathbb{P}(X \le x) $$ For reasons which I will never understand, some people plot $S(x) = 1 - F_X(x)$ for $F_X(x)$ the CDF of $X$, but call $S$ the CDF. It is completely baffling if you were taught that the CDF is non-decreasing. As with all conventions, it's ...


10

PMFs are associated with discrete random variables, PDFs with continuous random variables. For any type of random of random variable, the CDF always exists (and is unique), defined as $$F_X(x) = P\{X \leq x\}.$$ Now, depending on the support set of the random variable $X$, the density (or mass function) need not exist. (Consider the Cantor Set and Cantor ...


10

?dnorm is the density function for the normal distribution. If you enter a quantile (i.e., a value for X), and the mean and standard deviation of the normal distribution in question, it will output the probability density. ?pnorm is the distribution function for the normal distribution. If you enter a quantile, and the mean and standard deviation of the ...


10

Empirical is something you build from data and observations. For instance, suppose you want to know about the distribution of the height of people in a country. You start by measuring people and come up with a histogram that can be approximated to a distribution. Then you calculate the empirical CDF. If you are using a statistical distribution (a ...


10

As noted by Wikipedia, probability distribution function is ambiguous term: A probability distribution function is some function that may be used to define a particular probability distribution. Depending upon which text is consulted, the term may refer to: a cumulative distribution function, a probability mass function, and/or a ...


10

Problems like this, where you want to differentiate the product of a bunch of functions that depend on your variable of interest, can be dealt with by logarithmic differentiation. Let $\Phi$ and $\phi$ denote the CDF and PDF of the standard normal distribution (respectively). Since the normal random variables in your question have the same variance you get:...


Only top voted, non community-wiki answers of a minimum length are eligible