Questions tagged [ecdf]

Empirical cumulative distribution function: a step function increasing by $1/n$ at each unique $X$-value that occurred in the sample.

Filter by
Sorted by
Tagged with
0
votes
0answers
17 views

p value for the Cramér–von Mises two sample test in python [closed]

Where can I find a package that computes the p-value for the Cramér–von Mises 2 sample test in Python. I was able to find a package that computes the Cramér–von Mises 2 sample test itself : https://...
0
votes
0answers
21 views

Censored data - When does it matter

In survival analysis, one may arrive at a series of samples $X_1,...,X_n$, for which the outcome of a given $X$ may not be "observed" within the experiment. For instance, if the $X_i$'s are failure ...
3
votes
0answers
123 views

Comparing empirical distributions with an interaction effect

My experiment includes two subject groups: group1 and group2. Both groups undergo a behavioral test, the result of which is called “movement celerity” (this is my dependent variable). Both groups are ...
0
votes
1answer
17 views

Fit cdf and pdf from points on empirical cdf

I have cumulative counts with respect to a variable x, which looks like: ...
0
votes
0answers
28 views

Two-sample Kolmogorov–Smirnov w/ only relative population info

I have a set of experiments, each of which has an ECDF which represents my outcome. The number of samples used in each experiment is not the same. Unfortunately, I do not have information on the ...
2
votes
0answers
29 views

What are “absolute ECDFs” called (if anybody uses them)?

By absolute ECDF, I am referring to an ECDF that shows the absolute number of samples above some value as opposed to the fraction of such samples. For example, the bottom plot below displays such an ...
0
votes
2answers
48 views

Simulate data from ECDF in R [closed]

Consider the following 11 continuous non-negative observations: 0, 0.3, 0.31, 0.33, 0.37, 0.49, 0.51, 0.53, 0.59, 0.6 Obtain the empirical cumulative distribution function for these observations. ...
2
votes
1answer
32 views

How to quantify distribution concentration in cdf? [closed]

For example, let's say I have a list of users, each user has its revenue. I can plot the cdf of both user and revenue, to see if there is some concentration, for example, may be 40% user contribute ...
8
votes
3answers
347 views

Confidence Interval of CDF

I am trying to determine if there is a statistically meaningful distinction between the cumulative probability density curves shown in the figure below. It's simple enough to do a $t$-test on the ...
0
votes
0answers
26 views

How can I identify an unfamiliar cumulative distribution function?

I have 116 Bessel-corrected sample variances (average of squared distances from sample mean), each from a sample of three measurements. All measurements were done using the same method. I had ...
0
votes
1answer
32 views

Determine significance of an observed value

I have an ECDF of values that do not follow a particular distribution (thought they are slightly normal, they are not). And I wish to determine if a new observed value is significant or an outlier or ...
1
vote
2answers
124 views

Fitting a normal CDF using proportion data

I have the following data (prop is like empirical CDF): ...
0
votes
1answer
81 views

Is the Cumulative Hazard Function simply estimated using the Empirical CDF?

I'm just starting to learn about Survival Analysis and I understand the theoretical proofs but I'm still unclear about how to do some of the estimations practically.
2
votes
0answers
21 views

Under what conditions does the two-sided DKW inequality become a strict equality?

If the two-sided DKW inequality is tight, then there should be a choice of distribution and sample size where the equality holds. What is it?
0
votes
2answers
242 views

How does this code find the CDF?

How does the below code give CDF? Can someone please explain what np.arange(len(sorted_data))/float(len(sorted_data)-1) does? ...
3
votes
1answer
2k views

How to estimate probability density function (pdf) from empirical cumulative distribution function (ecdf)?

The context is survival analysis, where I have an empirical survival function in the form of a step function, which is just one minus the ecdf. Is there a standard way to get an estimate of the pdf (...
0
votes
0answers
53 views

How to calculate minimum sample size when using ecdf?

I am calculating the relative median income of census tracts with respect to the CBSA (essentially the metropolitan area) that contains them. I am using ecdf in R to get the percentiles for each set ...
4
votes
1answer
525 views

Looking for an unbiased version of the empirical cumulative distribution function that I can interpolate

Most definitions of the ECDF define it as (#elements <= threshold) / #elements. Matlab and R both implement their ecdf() functions using this formula. In my testing, however, I find that there is ...
3
votes
1answer
343 views

UMVUE of distribution function $F$ when $X_i\sim F$ are i.i.d random variables

Let $(X_1,X_2,\cdots,X_n)$ be a random sample drawn from a population with distribution function $F$. Is the empirical distribution function $F_n$ the UMVUE of $F$? ( $F$ itself is the parameter of ...
0
votes
0answers
22 views

How do I calculate the probability of something happening after a certain age?

I study a rare disease (recurrent respiratory papillomatosis (RRP)) and have accumulated data on hundreds of patients and the age at which they were diagnosed. Only a few (17) ever went on to develop ...
2
votes
0answers
74 views

Choice of bin width in empirical CDF

Non-parametric statistics literature (E.g. Eq 2.2 in here, or DKW inequality) define empirical CDF (ECDF) by assigning PMF of $\frac{1}{n}$ to each data point. They then proceed to make use of these ...
3
votes
1answer
194 views

Accuracy of empirical cumulative distribution function

I have a random variable with an unknown distribution and I want to find its cumulative distribution function. I sample the distribution $N$ times, with $$X_1, \dots, X_N$$ being iid random ...
2
votes
0answers
45 views

Interpret the KS test [duplicate]

I have two continuous distributions and I am trying to compare them using KS test using R Studio with the code below: ...
0
votes
1answer
294 views

How to estimate the margins distribution using ecdf?

Suppose we have a random variables x. x <- rnorm(500, 2,3) Suppose that we do not know the marginal distribution of (...
3
votes
1answer
834 views

Why do we need density in estimation and cumulative distribution in transformation?

In statistic, when we want to find the estimation of the model parameters, do we only work with the density function? Why do we need to work with density function instead of Cumulative distribution ...
0
votes
1answer
88 views

Is it Possible to Calculate Information Distances like the KL and Jensen-Shannon Divergences on EDFs and CDFs?

With the data I'm currently analyzing, it is easier to derive the EDFs than the PDFs; furthermore, I'm much more comfortable using CDF-EDF goodness-of-fit tests like the Kolmgorov-Smirnov and Anderson-...
2
votes
1answer
1k views

Monte Carlo Estimator for a quantile

I am trying to understand how to compute a crude Monte Carlo estimator for an $\alpha-$quantile. I have read the algorithm from the book Monte Carlo Methods and Models in Finance and Insurance, (Korn,...
0
votes
0answers
55 views

Reduce size of sample but remain CDF shape same as for original sample size

I can create cumulative distribution by issuing following steps (as explained in this question): ...
1
vote
2answers
1k views

Can the mean of an empirical CDF be different than .5?

I was asked to convert a series of values into a series of percentiles corresponding to these values with respect to the empirical cdf constituted by the initial series. Using R, I wrote: ...
1
vote
0answers
610 views

Input to fit a power-law to degree distribution of a network

I would like to use R to test whether the degree distribution of a network behave like a power-law with scale-free property. Nonetheless, I've read diferent people ...
5
votes
1answer
165 views

Can the Dvoretzky–Kiefer–Wolfowitz (a.k.a. Massart) inequality be extended to two empirical distributions?

Can the Dvoretzky–Kiefer–Wolfowitz (a.k.a. Massart) inequality $$ \Pr\left( \sup |\hat F_n(x) - F(x)| > \lambda \right) \le 2\exp(-2n\lambda^2) $$ where $\hat F_n(x)$ is the empirical ...
1
vote
1answer
412 views

ECDF for categorical data? [closed]

I have a data set of which contains the name of buyers. I want to plot a CDF that shows what percentage of buyers purchased up to what percentage of products. Something like bellow: , but I don't know ...
3
votes
1answer
1k views

How to Compute Bivariate Empirical Distribution?

Please can someone explain how to construct one with a little example. For instance let the pairs be $(X,Y)=\left\{(2,36),(8,12),(4,32)\right\}$ then what is $...
1
vote
0answers
58 views

Convergence of cumulative hazard type estimators

Suppose $\hat{\theta}_1=\int_0^x \frac {dF_n(y)} {1-F_n(y)}$ and $\hat{\theta}_2=\int_0^x \frac {f_n(y)} {1-F_n(y)} dy$ are two estimators of cumulative hazard function, where $f_n(y) =\frac {1} {b}\...
22
votes
5answers
19k views

Empirical CDF vs CDF

I'm learning about the Empirical Cumulative Distribution Function. But I still don't understand Why is it called 'Empirical'? Is there any difference between Empirical CDF and CDF?
2
votes
0answers
104 views

How to estimate density in a “more general” parametric method? (in R?)

As far as I know, when estimating the density of a distribution based on a sample from that distribution, we can take two extreme approached for estimation. The first is to not assume (almost) ...
4
votes
1answer
254 views

Two sample Kolmogorov-Smirnov test (How many times is D computed?)

If I want to conduct a two-sample Kolmogorov-Smirnov test - I understand I need to calculate the distance between two ECDFs multiple times across the length of the two curves, and then find the ...
2
votes
0answers
406 views

Plotting ECDF in R ggplot2, choosing a scale for the Y axis that shows maximum as well as 4-9's, 5-9's, etc

I regularly make graphs of ECDF for performance latencies, where I want to show an upper bound on latency at P(x) < 0.999, 0.9999, etc. As such, I usually scale the Y axis logarithmically to ...
0
votes
1answer
221 views

Population and the mean, standard deviation and the distribution of a population charactertistic

I have some questions regarding the statistical notion of "population'', the mean, standard deviation and the distribution of a population characteristic. Background: According to Wikipedia, ``A ...
13
votes
1answer
1k views

Why does ecdf uses a step function and not a linear interpolation?

Empirical CDF functions are usually estimated by a step function. Is there a reason why this is done in such a way and not by using a linear interpolation? Does the step function has any interesting ...
0
votes
1answer
158 views

Probability mass function from multiple datasets with different ranges

Assume 3 datasets (experiments) for the same population of a discrete random variable, dataset 1 has observed values {1, 2, 3, NA} values, dataset 2 {1, 2, 3, 4, 5, NA} and dataset 3 {1, 2, 3, 4, 5, 6,...
9
votes
2answers
3k views

Algorithms for computing multivariate Empirical distribution function (ECDF)?

One dimensional ECDF is fairly easy to compute. When it comes to two dimensions and up, however, online resources become sparse and hard to reach. Can anyone suggest, define and/or present efficient ...
3
votes
2answers
179 views

Is it better to look at ECDFs than PDFs when exploring empirical sample distributions?

I know that when plotting CDFs, ECDFs, or PDFs by using finite samples, we must be doing some form of interpolation. As far as I guess, empirical cumulative density functions (ECDFs) perform linear ...
3
votes
1answer
1k views

How can I get p-value by using ecdf and bootstrapping?

First of all, forgive my for my ignorance about this concept. I might ask a basic question but what I have read from Exploratory Data Analysis: 2 Ways of Plotting Empirical Cumulative Distribution ...
2
votes
2answers
2k views

Sampling from Empirical CDF for Forecasting

I'm trying to forecast the future distribution of a particular interest rate based on its quarterly percentage changes. My assumptions are that: The observations are independent The distribution ...
2
votes
1answer
2k views

Variance from cdf

I have an empirical cumulative probability distribution function $F$ for a random variable $X$ (non-negative). Is it possible to estimate the variance from $F$ ? I already estimated, numerically, the ...
1
vote
0answers
63 views

Area Based Kolmogorov Smirnov

I have several sets to two ECDFs that I want to compare. I know I can use the Kolmogorov-Smirnov test (and I have) but one set of ECDF is returning statistically significant when I suspect the ECDFs ...
7
votes
2answers
2k views

Comparing two ECDFs using Kolmogorov-Smirnov test (alternative hypothesis)

I am taking measurements of a computer system performance over time and I'd like to understand if the performance is degrading or improving as time passes.. After doing some research, I picked the KS ...
1
vote
0answers
56 views

Evaluating model performance - different possible metrics

I am comparing the root mean squared error (RMSE) of different regression models that were fit to simulated data. I have RMSE from 30 different simulated data sets for each regression model. Some of ...
1
vote
2answers
344 views

Density of an empirical cdf

I can figure how the underlying density function of an empirical-cdf looks like? Does it look like a histogram?