Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [ecdf]

Empirical cumulative distribution function: a step function increasing by $1/n$ at each unique $X$-value that occurred in the sample.

1
vote
0answers
12 views

Under what conditions does the two-sided DKW inequality become a strict equality?

If the two-sided DKW inequality is tight, then there should be a choice of distribution and sample size where the equality holds. What is it?
0
votes
0answers
77 views

Weighted Empirical Cumulative Distribution Function (CFD): order values before or after weighting? [before]

My input data are some observations of a continuous variable, let's say the number of kilometers of motorbikes. I did a survey and I collected some observations. Somehow, I computed weights for all ...
0
votes
2answers
58 views

How does this code find the CDF?

How does the below code give CDF? Can someone please explain what np.arange(len(sorted_data))/float(len(sorted_data)-1) does? ...
3
votes
1answer
709 views

How to estimate probability density function (pdf) from empirical cumulative distribution function (ecdf)?

The context is survival analysis, where I have an empirical survival function in the form of a step function, which is just one minus the ecdf. Is there a standard way to get an estimate of the pdf (...
1
vote
0answers
37 views

How to calculate minimum sample size when using ecdf?

I am calculating the relative median income of census tracts with respect to the CBSA (essentially the metropolitan area) that contains them. I am using ecdf in R to get the percentiles for each set ...
4
votes
1answer
338 views

Looking for an unbiased version of the empirical cumulative distribution function that I can interpolate

Most definitions of the ECDF define it as (#elements <= threshold) / #elements. Matlab and R both implement their ecdf() functions using this formula. In my testing, however, I find that there is ...
3
votes
1answer
193 views

UMVUE of distribution function $F$ when $X_i\sim F$ are i.i.d random variables

Let $(X_1,X_2,\cdots,X_n)$ be a random sample drawn from a population with distribution function $F$. Is the empirical distribution function $F_n$ the UMVUE of $F$? ( $F$ itself is the parameter of ...
0
votes
0answers
21 views

How do I calculate the probability of something happening after a certain age?

I study a rare disease (recurrent respiratory papillomatosis (RRP)) and have accumulated data on hundreds of patients and the age at which they were diagnosed. Only a few (17) ever went on to develop ...
2
votes
0answers
52 views

Choice of bin width in empirical CDF

Non-parametric statistics literature (E.g. Eq 2.2 in here, or DKW inequality) define empirical CDF (ECDF) by assigning PMF of $\frac{1}{n}$ to each data point. They then proceed to make use of these ...
3
votes
1answer
117 views

Accuracy of empirical cumulative distribution function

I have a random variable with an unknown distribution and I want to find its cumulative distribution function. I sample the distribution $N$ times, with $$X_1, \dots, X_N$$ being iid random ...
2
votes
0answers
44 views

Interpret the KS test [duplicate]

I have two continuous distributions and I am trying to compare them using KS test using R Studio with the code below: ...
0
votes
1answer
179 views

How to estimate the margins distribution using ecdf?

Suppose we have a random variables x. x <- rnorm(500, 2,3) Suppose that we do not know the marginal distribution of (...
3
votes
1answer
564 views

Why do we need density in estimation and cumulative distribution in transformation?

In statistic, when we want to find the estimation of the model parameters, do we only work with the density function? Why do we need to work with density function instead of Cumulative distribution ...
0
votes
1answer
67 views

Is it Possible to Calculate Information Distances like the KL and Jensen-Shannon Divergences on EDFs and CDFs?

With the data I'm currently analyzing, it is easier to derive the EDFs than the PDFs; furthermore, I'm much more comfortable using CDF-EDF goodness-of-fit tests like the Kolmgorov-Smirnov and Anderson-...
2
votes
1answer
751 views

Monte Carlo Estimator for a quantile

I am trying to understand how to compute a crude Monte Carlo estimator for an $\alpha-$quantile. I have read the algorithm from the book Monte Carlo Methods and Models in Finance and Insurance, (Korn,...
0
votes
0answers
41 views

Reduce size of sample but remain CDF shape same as for original sample size

I can create cumulative distribution by issuing following steps (as explained in this question): ...
1
vote
2answers
739 views

Can the mean of an empirical CDF be different than .5?

I was asked to convert a series of values into a series of percentiles corresponding to these values with respect to the empirical cdf constituted by the initial series. Using R, I wrote: ...
1
vote
0answers
451 views

Input to fit a power-law to degree distribution of a network

I would like to use R to test whether the degree distribution of a network behave like a power-law with scale-free property. Nonetheless, I've read diferent people ...
5
votes
1answer
137 views

Can the Dvoretzky–Kiefer–Wolfowitz (a.k.a. Massart) inequality be extended to two empirical distributions?

Can the Dvoretzky–Kiefer–Wolfowitz (a.k.a. Massart) inequality $$ \Pr\left( \sup |\hat F_n(x) - F(x)| > \lambda \right) \le 2\exp(-2n\lambda^2) $$ where $\hat F_n(x)$ is the empirical ...
1
vote
1answer
207 views

ECDF for categorical data? [closed]

I have a data set of which contains the name of buyers. I want to plot a CDF that shows what percentage of buyers purchased up to what percentage of products. Something like bellow: , but I don't know ...
3
votes
1answer
677 views

How to Compute Bivariate Empirical Distribution?

Please can someone explain how to construct one with a little example. For instance let the pairs be $(X,Y)=\left\{(2,36),(8,12),(4,32)\right\}$ then what is $...
1
vote
0answers
57 views

Convergence of cumulative hazard type estimators

Suppose $\hat{\theta}_1=\int_0^x \frac {dF_n(y)} {1-F_n(y)}$ and $\hat{\theta}_2=\int_0^x \frac {f_n(y)} {1-F_n(y)} dy$ are two estimators of cumulative hazard function, where $f_n(y) =\frac {1} {b}\...
17
votes
5answers
14k views

Empirical CDF vs CDF

I'm learning about the Empirical Cumulative Distribution Function. But I still don't understand Why is it called 'Empirical'? Is there any difference between Empirical CDF and CDF?
2
votes
0answers
90 views

How to estimate density in a “more general” parametric method? (in R?)

As far as I know, when estimating the density of a distribution based on a sample from that distribution, we can take two extreme approached for estimation. The first is to not assume (almost) ...
4
votes
1answer
224 views

Two sample Kolmogorov-Smirnov test (How many times is D computed?)

If I want to conduct a two-sample Kolmogorov-Smirnov test - I understand I need to calculate the distance between two ECDFs multiple times across the length of the two curves, and then find the ...
2
votes
0answers
326 views

Plotting ECDF in R ggplot2, choosing a scale for the Y axis that shows maximum as well as 4-9's, 5-9's, etc

I regularly make graphs of ECDF for performance latencies, where I want to show an upper bound on latency at P(x) < 0.999, 0.9999, etc. As such, I usually scale the Y axis logarithmically to ...
0
votes
1answer
172 views

Population and the mean, standard deviation and the distribution of a population charactertistic

I have some questions regarding the statistical notion of "population'', the mean, standard deviation and the distribution of a population characteristic. Background: According to Wikipedia, ``A ...
12
votes
1answer
1k views

Why does ecdf uses a step function and not a linear interpolation?

Empirical CDF functions are usually estimated by a step function. Is there a reason why this is done in such a way and not by using a linear interpolation? Does the step function has any interesting ...
0
votes
1answer
110 views

Probability mass function from multiple datasets with different ranges

Assume 3 datasets (experiments) for the same population of a discrete random variable, dataset 1 has observed values {1, 2, 3, NA} values, dataset 2 {1, 2, 3, 4, 5, NA} and dataset 3 {1, 2, 3, 4, 5, 6,...
7
votes
2answers
2k views

Algorithms for computing multivariate Empirical distribution function (ECDF)?

One dimensional ECDF is fairly easy to compute. When it comes to two dimensions and up, however, online resources become sparse and hard to reach. Can anyone suggest, define and/or present efficient ...
2
votes
2answers
114 views

Is it better to look at ECDFs than PDFs when exploring empirical sample distributions?

I know that when plotting CDFs, ECDFs, or PDFs by using finite samples, we must be doing some form of interpolation. As far as I guess, empirical cumulative density functions (ECDFs) perform linear ...
3
votes
1answer
916 views

How can I get p-value by using ecdf and bootstrapping?

First of all, forgive my for my ignorance about this concept. I might ask a basic question but what I have read from Exploratory Data Analysis: 2 Ways of Plotting Empirical Cumulative Distribution ...
2
votes
2answers
2k views

Sampling from Empirical CDF for Forecasting

I'm trying to forecast the future distribution of a particular interest rate based on its quarterly percentage changes. My assumptions are that: The observations are independent The distribution ...
2
votes
1answer
777 views

Variance from cdf

I have an empirical cumulative probability distribution function $F$ for a random variable $X$ (non-negative). Is it possible to estimate the variance from $F$ ? I already estimated, numerically, the ...
1
vote
0answers
58 views

Area Based Kolmogorov Smirnov

I have several sets to two ECDFs that I want to compare. I know I can use the Kolmogorov-Smirnov test (and I have) but one set of ECDF is returning statistically significant when I suspect the ECDFs ...
7
votes
0answers
2k views

Comparing two ECDFs using Kolmogorov-Smirnov test (alternative hypothesis)

I am taking measurements of a computer system performance over time and I'd like to understand if the performance is degrading or improving as time passes.. After doing some research, I picked the KS ...
1
vote
0answers
53 views

Evaluating model performance - different possible metrics

I am comparing the root mean squared error (RMSE) of different regression models that were fit to simulated data. I have RMSE from 30 different simulated data sets for each regression model. Some of ...
1
vote
2answers
242 views

Density of an empirical cdf

I can figure how the underlying density function of an empirical-cdf looks like? Does it look like a histogram?
4
votes
1answer
763 views

Estimating a cumulative distribution function from a mixture model

I have a problem I fail to find a solution. Assume a mixture model : $F(x)=a\times G(x)+(1-a)\times H(x)$ where $F$, $G$ and $H$ are cumulative distributions of different random variables, and $a$ ...
0
votes
1answer
540 views

Calculate probability of event from ecdf in R

I'm using R and would like to calculate the probability of an event happening. I used the ecdf function on daily revenue (month-to-date). I have a target and a daily number necessary to hit target ...
6
votes
0answers
325 views

Empirical distribution function of overlapping time series data

If we model asset return volatility for periods of more than one (say more than one day) there is the square-root rule which holds true under some assumptions. On the other hand practitioners ...
2
votes
0answers
42 views

Describing Differences between Two Inferred Populations

I am interested in the size of an animal in two populations, and found reliable estimates of the population provided by NOAA. The data was in the form of estimated total population within discrete ...
2
votes
1answer
954 views

How to compute joint cdf of an empirical copula? (Updated with more info)

lets suppose a bivariate empirical copula as: for a set of data of example data we can plot it like this: How can we compute the joint cdf of this empirical copula which should like this: Thank you
0
votes
1answer
208 views

When to use empirical cdf?

I saw in some papers that people sometimes use empirical cdf or complementary cdf to quantify the distribution of data. What is the advantage of using them and what interesting can they tell about ...
2
votes
2answers
720 views

What does a Barplot, a Boxplot and eCDF represent?

I want to understand what a barplot, boxplot, and eCDF represent when they are used for plotting data. A barplot is kind of intuitive, easy to understand what it represents. But I fail to understand ...
1
vote
1answer
1k views

Convergence in distribution with empirical distribution function (EDF)

I'm struggling with the following question: Let $F_n(x)$ denote the EDF of a random sample. Show that $\sqrt{n}(F_n(x)-F(x))\xrightarrow[]{d}N(0,F(x)(1-F(x)))$. I think that the course of ...
3
votes
1answer
183 views

Results stronger than Dvoretzky–Kiefer–Wolfowitz inequality?

There is the DKW inequality which controls the extent to which the empirical cdf of a sample from a real-valued random variable differs from the true cdf. Are there any stronger results (i.e. ...
1
vote
1answer
662 views

CDF for list of numbers

I hope that someone can help me with this! I have a list of values as below: 82.1134 84.5516 91.1851 65.6035 69.971 92.4706 79.1505 93.0844 92.9598 and I need to ...
3
votes
0answers
52 views

Verification of an optimal parameter from an empirical CDF

Suppose we have the following model for the variable $V_5$: $$V_5 = \prod_{k=1}^5(e^{\mu + 0.2X_k}+0.05e^{0.05Y_i - 0.00125}), X_i,Y_i\sim N(0,1)$$ What I wish to do is to solve the problem $\min_{\...
5
votes
2answers
503 views

Fitting parametric CDF to ecdf

There is a random variable $X$, but the only data I observed is actually its empirical distribution function (at a suitably fine grid). That is, I only observe $\hat{F}(x)$:=$\#\{x\leq u\}\over N $ ...