Questions tagged [ecdf]

Empirical cumulative distribution function: a step function increasing by $1/n$ at each unique $X$-value that occurred in the sample.

66 questions
Filter by
Sorted by
Tagged with
22 views

Determine significance of an observed value

I have an ECDF of values that do not follow a particular distribution (thought they are slightly normal, they are not). And I wish to determine if a new observed value is significant or an outlier or ...
69 views

Fitting a normal CDF using proportion data

I have the following data (prop is like empirical CDF): ...
34 views

Is the Cumulative Hazard Function simply estimated using the Empirical CDF?

I'm just starting to learn about Survival Analysis and I understand the theoretical proofs but I'm still unclear about how to do some of the estimations practically.
15 views

Under what conditions does the two-sided DKW inequality become a strict equality?

If the two-sided DKW inequality is tight, then there should be a choice of distribution and sample size where the equality holds. What is it?
108 views

Weighted Empirical Cumulative Distribution Function (CFD): order values before or after weighting? [before]

My input data are some observations of a continuous variable, let's say the number of kilometers of motorbikes. I did a survey and I collected some observations. Somehow, I computed weights for all ...
101 views

How does this code find the CDF?

How does the below code give CDF? Can someone please explain what np.arange(len(sorted_data))/float(len(sorted_data)-1) does? ...
910 views

How to estimate probability density function (pdf) from empirical cumulative distribution function (ecdf)?

The context is survival analysis, where I have an empirical survival function in the form of a step function, which is just one minus the ecdf. Is there a standard way to get an estimate of the pdf (...
43 views

How to calculate minimum sample size when using ecdf?

I am calculating the relative median income of census tracts with respect to the CBSA (essentially the metropolitan area) that contains them. I am using ecdf in R to get the percentiles for each set ...
399 views

Looking for an unbiased version of the empirical cumulative distribution function that I can interpolate

Most definitions of the ECDF define it as (#elements <= threshold) / #elements. Matlab and R both implement their ecdf() functions using this formula. In my testing, however, I find that there is ...
217 views

UMVUE of distribution function $F$ when $X_i\sim F$ are i.i.d random variables

Let $(X_1,X_2,\cdots,X_n)$ be a random sample drawn from a population with distribution function $F$. Is the empirical distribution function $F_n$ the UMVUE of $F$? ( $F$ itself is the parameter of ...
21 views

How do I calculate the probability of something happening after a certain age?

I study a rare disease (recurrent respiratory papillomatosis (RRP)) and have accumulated data on hundreds of patients and the age at which they were diagnosed. Only a few (17) ever went on to develop ...
56 views

Choice of bin width in empirical CDF

Non-parametric statistics literature (E.g. Eq 2.2 in here, or DKW inequality) define empirical CDF (ECDF) by assigning PMF of $\frac{1}{n}$ to each data point. They then proceed to make use of these ...
153 views

Accuracy of empirical cumulative distribution function

I have a random variable with an unknown distribution and I want to find its cumulative distribution function. I sample the distribution $N$ times, with $$X_1, \dots, X_N$$ being iid random ...
45 views

Interpret the KS test [duplicate]

I have two continuous distributions and I am trying to compare them using KS test using R Studio with the code below: ...
217 views

How to estimate the margins distribution using ecdf?

Suppose we have a random variables x. x <- rnorm(500, 2,3) Suppose that we do not know the marginal distribution of (...
661 views

Why do we need density in estimation and cumulative distribution in transformation?

In statistic, when we want to find the estimation of the model parameters, do we only work with the density function? Why do we need to work with density function instead of Cumulative distribution ...
72 views

Is it Possible to Calculate Information Distances like the KL and Jensen-Shannon Divergences on EDFs and CDFs?

With the data I'm currently analyzing, it is easier to derive the EDFs than the PDFs; furthermore, I'm much more comfortable using CDF-EDF goodness-of-fit tests like the Kolmgorov-Smirnov and Anderson-...
878 views

Monte Carlo Estimator for a quantile

I am trying to understand how to compute a crude Monte Carlo estimator for an $\alpha-$quantile. I have read the algorithm from the book Monte Carlo Methods and Models in Finance and Insurance, (Korn,...
47 views

Reduce size of sample but remain CDF shape same as for original sample size

I can create cumulative distribution by issuing following steps (as explained in this question): ...
819 views

Can the mean of an empirical CDF be different than .5?

I was asked to convert a series of values into a series of percentiles corresponding to these values with respect to the empirical cdf constituted by the initial series. Using R, I wrote: ...
485 views

Input to fit a power-law to degree distribution of a network

I would like to use R to test whether the degree distribution of a network behave like a power-law with scale-free property. Nonetheless, I've read diferent people ...
151 views

Can the Dvoretzky–Kiefer–Wolfowitz (a.k.a. Massart) inequality be extended to two empirical distributions?

Can the Dvoretzky–Kiefer–Wolfowitz (a.k.a. Massart) inequality $$\Pr\left( \sup |\hat F_n(x) - F(x)| > \lambda \right) \le 2\exp(-2n\lambda^2)$$ where $\hat F_n(x)$ is the empirical ...
264 views

ECDF for categorical data? [closed]

I have a data set of which contains the name of buyers. I want to plot a CDF that shows what percentage of buyers purchased up to what percentage of products. Something like bellow: , but I don't know ...
768 views

15k views

Empirical CDF vs CDF

I'm learning about the Empirical Cumulative Distribution Function. But I still don't understand Why is it called 'Empirical'? Is there any difference between Empirical CDF and CDF?
93 views

How to estimate density in a “more general” parametric method? (in R?)

As far as I know, when estimating the density of a distribution based on a sample from that distribution, we can take two extreme approached for estimation. The first is to not assume (almost) ...
239 views

Two sample Kolmogorov-Smirnov test (How many times is D computed?)

If I want to conduct a two-sample Kolmogorov-Smirnov test - I understand I need to calculate the distance between two ECDFs multiple times across the length of the two curves, and then find the ...
358 views

Plotting ECDF in R ggplot2, choosing a scale for the Y axis that shows maximum as well as 4-9's, 5-9's, etc

I regularly make graphs of ECDF for performance latencies, where I want to show an upper bound on latency at P(x) < 0.999, 0.9999, etc. As such, I usually scale the Y axis logarithmically to ...
181 views

Population and the mean, standard deviation and the distribution of a population charactertistic

I have some questions regarding the statistical notion of "population'', the mean, standard deviation and the distribution of a population characteristic. Background: According to Wikipedia, A ...
1k views

Why does ecdf uses a step function and not a linear interpolation?

Empirical CDF functions are usually estimated by a step function. Is there a reason why this is done in such a way and not by using a linear interpolation? Does the step function has any interesting ...
128 views

Probability mass function from multiple datasets with different ranges

Assume 3 datasets (experiments) for the same population of a discrete random variable, dataset 1 has observed values {1, 2, 3, NA} values, dataset 2 {1, 2, 3, 4, 5, NA} and dataset 3 {1, 2, 3, 4, 5, 6,...
2k views

Algorithms for computing multivariate Empirical distribution function (ECDF)?

One dimensional ECDF is fairly easy to compute. When it comes to two dimensions and up, however, online resources become sparse and hard to reach. Can anyone suggest, define and/or present efficient ...
130 views

Is it better to look at ECDFs than PDFs when exploring empirical sample distributions?

I know that when plotting CDFs, ECDFs, or PDFs by using finite samples, we must be doing some form of interpolation. As far as I guess, empirical cumulative density functions (ECDFs) perform linear ...
1k views

How can I get p-value by using ecdf and bootstrapping?

First of all, forgive my for my ignorance about this concept. I might ask a basic question but what I have read from Exploratory Data Analysis: 2 Ways of Plotting Empirical Cumulative Distribution ...
2k views

Sampling from Empirical CDF for Forecasting

I'm trying to forecast the future distribution of a particular interest rate based on its quarterly percentage changes. My assumptions are that: The observations are independent The distribution ...
1k views

Variance from cdf

I have an empirical cumulative probability distribution function $F$ for a random variable $X$ (non-negative). Is it possible to estimate the variance from $F$ ? I already estimated, numerically, the ...
59 views

Area Based Kolmogorov Smirnov

I have several sets to two ECDFs that I want to compare. I know I can use the Kolmogorov-Smirnov test (and I have) but one set of ECDF is returning statistically significant when I suspect the ECDFs ...
2k views

Comparing two ECDFs using Kolmogorov-Smirnov test (alternative hypothesis)

I am taking measurements of a computer system performance over time and I'd like to understand if the performance is degrading or improving as time passes.. After doing some research, I picked the KS ...
55 views

Evaluating model performance - different possible metrics

I am comparing the root mean squared error (RMSE) of different regression models that were fit to simulated data. I have RMSE from 30 different simulated data sets for each regression model. Some of ...
275 views

Density of an empirical cdf

I can figure how the underlying density function of an empirical-cdf looks like? Does it look like a histogram?
783 views

Estimating a cumulative distribution function from a mixture model

I have a problem I fail to find a solution. Assume a mixture model : $F(x)=a\times G(x)+(1-a)\times H(x)$ where $F$, $G$ and $H$ are cumulative distributions of different random variables, and $a$ ...
579 views

Calculate probability of event from ecdf in R

I'm using R and would like to calculate the probability of an event happening. I used the ecdf function on daily revenue (month-to-date). I have a target and a daily number necessary to hit target ...
335 views

Empirical distribution function of overlapping time series data

If we model asset return volatility for periods of more than one (say more than one day) there is the square-root rule which holds true under some assumptions. On the other hand practitioners ...
42 views

Describing Differences between Two Inferred Populations

I am interested in the size of an animal in two populations, and found reliable estimates of the population provided by NOAA. The data was in the form of estimated total population within discrete ...
1k views

How to compute joint cdf of an empirical copula? (Updated with more info)

lets suppose a bivariate empirical copula as: for a set of data of example data we can plot it like this: How can we compute the joint cdf of this empirical copula which should like this: Thank you
227 views

When to use empirical cdf?

I saw in some papers that people sometimes use empirical cdf or complementary cdf to quantify the distribution of data. What is the advantage of using them and what interesting can they tell about ...
762 views

What does a Barplot, a Boxplot and eCDF represent?

I want to understand what a barplot, boxplot, and eCDF represent when they are used for plotting data. A barplot is kind of intuitive, easy to understand what it represents. But I fail to understand ...
I'm struggling with the following question: Let $F_n(x)$ denote the EDF of a random sample. Show that $\sqrt{n}(F_n(x)-F(x))\xrightarrow[]{d}N(0,F(x)(1-F(x)))$. I think that the course of ...