I'm learning about the Empirical Cumulative Distribution Function. But I still don't understand
Why is it called 'Empirical'?
Is there any difference between Empirical CDF and CDF?
$\begingroup$
$\endgroup$
3
-
2$\begingroup$ Check here stats.stackexchange.com/questions/222120/… $\endgroup$– TimCommented Oct 13, 2016 at 6:11
-
1$\begingroup$ There is a simple, straightforward, elegant explanation in terms of tickets in a box models: the CDF describes what is in the original box. The ECDF is what you get when you put your sample (which is a set of tickets drawn from the original box: so-called "empirical" data) into an empty box. $\endgroup$– whuber ♦Commented Oct 13, 2016 at 18:57
-
$\begingroup$ One thing to be aware of is that your empirical distribution is usually bounded by the way it's constructed, while the CDF may not be. For instance, if you build empirical CDF from observations of Poisson variable, the obtained ECDF is going to be bounded by the highest observed frequency, while the true CDF is unbounded. $\endgroup$– AksakalCommented Oct 17, 2016 at 17:37
Add a comment
|
5 Answers
$\begingroup$
$\endgroup$
Let $X$ be a random variable.
- The cumulative distribution function $F(x)$ gives the $P(X \leq x)$.
- An empirical cumulative distribution function function $G(x)$ gives $P(X \leq x)$ based on the observations in your sample.
The distinction is which probability measure is used. For the empirical CDF, you use the probability measure defined by the frequency counts in an empirical sample.
Simple example (coin flip):
Let $X$ be a random variable denoting the result of a single coin flip where $X=1$ denotes heads and $X=0$ denotes tails.
The CDF for a fair coin is given by: $$ F(x) = \left\{ \begin{array}{ll} 0 & \text{for } x < 0\\ \frac{1}{2} & \text{for } 0 \leq x < 1 \\1 & \text{for } 1 \leq x \end{array} \right. $$
If you flipped 2 heads and 1 tail, the empirical CDF would be: $$ G(x) = \left\{ \begin{array}{ll} 0 & \text{for } x < 0\\ \frac{2}{3} & \text{for } 0 \leq x < 1 \\1 & \text{for } 1 \leq x \end{array} \right. $$
The empirical CDF would reflect that in your sample, $2/3$ of your flips were heads.
Another example ($F$ is CDF for normal distribution):
Let $X$ be a normally distributed random variable with mean $0$ and standard deviation $1$.
The CDF is given by:
$$F(x) = \int_{-\infty}^x \frac{1}{\sqrt{2\pi}} e^{\frac{-x^2}{2}}$$
Let's say you had 3 IID draws and obtained the values $x_1 < x_2 < x_3$. The empirical CDF would be: $$ G(y) = \left\{ \begin{array}{ll} 0 & \text{for } y < x_1\\ \frac{1}{3} & \text{for } x_1 \leq y < x_2 \\\frac{2}{3} & \text{for } x_2 \leq y < x_3 \\1 & \text{for } x_3 \leq y \end{array} \right. $$
With enough IID draws (and certain regularity conditions are satisfied), the empirical CDF would converge on the underlying CDF of the population.
answered Oct 13, 2016 at 6:36
$\begingroup$
$\endgroup$
Is there any difference between Empirical CDF and CDF?
Yes, they're different. An empirical cdf is a proper cdf, but empirical cdfs will always be discrete even when not drawn from a discrete distribution, while the cdf of a distribution can be other things besides discrete.
If you treat a sample as if it were a population of values, each one equally probable (i.e. place probability 1/n on each observation) then the cdf of that distribution would be the ECDF of the data.
Why does it called 'Empirical'?
It's an estimate of the population cdf based on the sample; specifically if you treat the proportions of the sample at each distinct data value and treat it like it was a probability in the population, you get the ECDF.
Empirical has a meaning something like "by observation rather than theory", and that's exactly what it means in this case ... using the observations to determine the distribution function.
$\begingroup$
$\endgroup$
The empirical CDF is built from an actual data set (in the plot below, I used 100 samples from a standard normal distribution). The CDF is a theoretical construct - it is what you would see if you could take infinitely many samples.
The empirical CDF usually approximates the CDF quite well, especially for large samples (in fact, there are theorems about how quickly it converges to the CDF as the sample size increases).
answered Oct 14, 2016 at 7:30
$\begingroup$
$\endgroup$
1
Empirical is something you build from data and observations. For instance, suppose you want to know about the distribution of the height of people in a country. You start by measuring people and come up with a histogram that can be approximated to a distribution. Then you calculate the empirical CDF.
If you are using a statistical distribution (a deterministic formula that gives the exact same output with the same parameters) you can calculate its CDF also.
You can say "The height of the people in this country is distributed similar to normal distribution with the mean 1.75 m and the standard deviation 0.1 m. Then you can use CDF of ~$N(\mu=1.75\ \text{m},\sigma=0.1\ \text{m})$ instead of the constructed CDF of the empirical distribution.
-
$\begingroup$ Is there a confidence measurement employed that expresses the likelihood that CDF and Emperical CDF describe the same population in the limit of all the experimental sampling in the world? This would seem to have application to Electoral polling, for instance. (though maybe not, since the output is not strictly describable as a function...) $\endgroup$– BenPenCommented Oct 13, 2016 at 15:00
$\begingroup$
$\endgroup$
According to Dictionary.com, the definitions of "empirical" include:
derived from or guided by experience or experiment.
Hence, the Empirical CDF is the CDF you obtain from your data. This contrasts with the theoretical CDF (often just called "CDF"), which is obtained from a statistical or probabilistic model such as the Normal distribution.
answered Oct 13, 2016 at 6:34