Questions tagged [ecdf]

Empirical cumulative distribution function: a step function increasing by $1/n$ at each unique $X$-value that occurred in the sample.

Filter by
Sorted by
Tagged with
0
votes
1answer
22 views

Determine significance of an observed value

I have an ECDF of values that do not follow a particular distribution (thought they are slightly normal, they are not). And I wish to determine if a new observed value is significant or an outlier or ...
1
vote
2answers
69 views

Fitting a normal CDF using proportion data

I have the following data (prop is like empirical CDF): ...
0
votes
1answer
34 views

Is the Cumulative Hazard Function simply estimated using the Empirical CDF?

I'm just starting to learn about Survival Analysis and I understand the theoretical proofs but I'm still unclear about how to do some of the estimations practically.
1
vote
0answers
15 views

Under what conditions does the two-sided DKW inequality become a strict equality?

If the two-sided DKW inequality is tight, then there should be a choice of distribution and sample size where the equality holds. What is it?
0
votes
0answers
108 views

Weighted Empirical Cumulative Distribution Function (CFD): order values before or after weighting? [before]

My input data are some observations of a continuous variable, let's say the number of kilometers of motorbikes. I did a survey and I collected some observations. Somehow, I computed weights for all ...
0
votes
2answers
101 views

How does this code find the CDF?

How does the below code give CDF? Can someone please explain what np.arange(len(sorted_data))/float(len(sorted_data)-1) does? ...
3
votes
1answer
910 views

How to estimate probability density function (pdf) from empirical cumulative distribution function (ecdf)?

The context is survival analysis, where I have an empirical survival function in the form of a step function, which is just one minus the ecdf. Is there a standard way to get an estimate of the pdf (...
1
vote
0answers
43 views

How to calculate minimum sample size when using ecdf?

I am calculating the relative median income of census tracts with respect to the CBSA (essentially the metropolitan area) that contains them. I am using ecdf in R to get the percentiles for each set ...
4
votes
1answer
399 views

Looking for an unbiased version of the empirical cumulative distribution function that I can interpolate

Most definitions of the ECDF define it as (#elements <= threshold) / #elements. Matlab and R both implement their ecdf() functions using this formula. In my testing, however, I find that there is ...
3
votes
1answer
217 views

UMVUE of distribution function $F$ when $X_i\sim F$ are i.i.d random variables

Let $(X_1,X_2,\cdots,X_n)$ be a random sample drawn from a population with distribution function $F$. Is the empirical distribution function $F_n$ the UMVUE of $F$? ( $F$ itself is the parameter of ...
0
votes
0answers
21 views

How do I calculate the probability of something happening after a certain age?

I study a rare disease (recurrent respiratory papillomatosis (RRP)) and have accumulated data on hundreds of patients and the age at which they were diagnosed. Only a few (17) ever went on to develop ...
2
votes
0answers
56 views

Choice of bin width in empirical CDF

Non-parametric statistics literature (E.g. Eq 2.2 in here, or DKW inequality) define empirical CDF (ECDF) by assigning PMF of $\frac{1}{n}$ to each data point. They then proceed to make use of these ...
3
votes
1answer
153 views

Accuracy of empirical cumulative distribution function

I have a random variable with an unknown distribution and I want to find its cumulative distribution function. I sample the distribution $N$ times, with $$X_1, \dots, X_N$$ being iid random ...
2
votes
0answers
45 views

Interpret the KS test [duplicate]

I have two continuous distributions and I am trying to compare them using KS test using R Studio with the code below: ...
0
votes
1answer
217 views

How to estimate the margins distribution using ecdf?

Suppose we have a random variables x. x <- rnorm(500, 2,3) Suppose that we do not know the marginal distribution of (...
3
votes
1answer
661 views

Why do we need density in estimation and cumulative distribution in transformation?

In statistic, when we want to find the estimation of the model parameters, do we only work with the density function? Why do we need to work with density function instead of Cumulative distribution ...
0
votes
1answer
72 views

Is it Possible to Calculate Information Distances like the KL and Jensen-Shannon Divergences on EDFs and CDFs?

With the data I'm currently analyzing, it is easier to derive the EDFs than the PDFs; furthermore, I'm much more comfortable using CDF-EDF goodness-of-fit tests like the Kolmgorov-Smirnov and Anderson-...
2
votes
1answer
878 views

Monte Carlo Estimator for a quantile

I am trying to understand how to compute a crude Monte Carlo estimator for an $\alpha-$quantile. I have read the algorithm from the book Monte Carlo Methods and Models in Finance and Insurance, (Korn,...
0
votes
0answers
47 views

Reduce size of sample but remain CDF shape same as for original sample size

I can create cumulative distribution by issuing following steps (as explained in this question): ...
1
vote
2answers
819 views

Can the mean of an empirical CDF be different than .5?

I was asked to convert a series of values into a series of percentiles corresponding to these values with respect to the empirical cdf constituted by the initial series. Using R, I wrote: ...
1
vote
0answers
485 views

Input to fit a power-law to degree distribution of a network

I would like to use R to test whether the degree distribution of a network behave like a power-law with scale-free property. Nonetheless, I've read diferent people ...
5
votes
1answer
151 views

Can the Dvoretzky–Kiefer–Wolfowitz (a.k.a. Massart) inequality be extended to two empirical distributions?

Can the Dvoretzky–Kiefer–Wolfowitz (a.k.a. Massart) inequality $$ \Pr\left( \sup |\hat F_n(x) - F(x)| > \lambda \right) \le 2\exp(-2n\lambda^2) $$ where $\hat F_n(x)$ is the empirical ...
1
vote
1answer
264 views

ECDF for categorical data? [closed]

I have a data set of which contains the name of buyers. I want to plot a CDF that shows what percentage of buyers purchased up to what percentage of products. Something like bellow: , but I don't know ...
3
votes
1answer
768 views

How to Compute Bivariate Empirical Distribution?

Please can someone explain how to construct one with a little example. For instance let the pairs be $(X,Y)=\left\{(2,36),(8,12),(4,32)\right\}$ then what is $...
1
vote
0answers
58 views

Convergence of cumulative hazard type estimators

Suppose $\hat{\theta}_1=\int_0^x \frac {dF_n(y)} {1-F_n(y)}$ and $\hat{\theta}_2=\int_0^x \frac {f_n(y)} {1-F_n(y)} dy$ are two estimators of cumulative hazard function, where $f_n(y) =\frac {1} {b}\...
18
votes
5answers
15k views

Empirical CDF vs CDF

I'm learning about the Empirical Cumulative Distribution Function. But I still don't understand Why is it called 'Empirical'? Is there any difference between Empirical CDF and CDF?
2
votes
0answers
93 views

How to estimate density in a “more general” parametric method? (in R?)

As far as I know, when estimating the density of a distribution based on a sample from that distribution, we can take two extreme approached for estimation. The first is to not assume (almost) ...
4
votes
1answer
239 views

Two sample Kolmogorov-Smirnov test (How many times is D computed?)

If I want to conduct a two-sample Kolmogorov-Smirnov test - I understand I need to calculate the distance between two ECDFs multiple times across the length of the two curves, and then find the ...
2
votes
0answers
358 views

Plotting ECDF in R ggplot2, choosing a scale for the Y axis that shows maximum as well as 4-9's, 5-9's, etc

I regularly make graphs of ECDF for performance latencies, where I want to show an upper bound on latency at P(x) < 0.999, 0.9999, etc. As such, I usually scale the Y axis logarithmically to ...
0
votes
1answer
181 views

Population and the mean, standard deviation and the distribution of a population charactertistic

I have some questions regarding the statistical notion of "population'', the mean, standard deviation and the distribution of a population characteristic. Background: According to Wikipedia, ``A ...
12
votes
1answer
1k views

Why does ecdf uses a step function and not a linear interpolation?

Empirical CDF functions are usually estimated by a step function. Is there a reason why this is done in such a way and not by using a linear interpolation? Does the step function has any interesting ...
0
votes
1answer
128 views

Probability mass function from multiple datasets with different ranges

Assume 3 datasets (experiments) for the same population of a discrete random variable, dataset 1 has observed values {1, 2, 3, NA} values, dataset 2 {1, 2, 3, 4, 5, NA} and dataset 3 {1, 2, 3, 4, 5, 6,...
7
votes
2answers
2k views

Algorithms for computing multivariate Empirical distribution function (ECDF)?

One dimensional ECDF is fairly easy to compute. When it comes to two dimensions and up, however, online resources become sparse and hard to reach. Can anyone suggest, define and/or present efficient ...
2
votes
2answers
130 views

Is it better to look at ECDFs than PDFs when exploring empirical sample distributions?

I know that when plotting CDFs, ECDFs, or PDFs by using finite samples, we must be doing some form of interpolation. As far as I guess, empirical cumulative density functions (ECDFs) perform linear ...
3
votes
1answer
1k views

How can I get p-value by using ecdf and bootstrapping?

First of all, forgive my for my ignorance about this concept. I might ask a basic question but what I have read from Exploratory Data Analysis: 2 Ways of Plotting Empirical Cumulative Distribution ...
2
votes
2answers
2k views

Sampling from Empirical CDF for Forecasting

I'm trying to forecast the future distribution of a particular interest rate based on its quarterly percentage changes. My assumptions are that: The observations are independent The distribution ...
2
votes
1answer
1k views

Variance from cdf

I have an empirical cumulative probability distribution function $F$ for a random variable $X$ (non-negative). Is it possible to estimate the variance from $F$ ? I already estimated, numerically, the ...
1
vote
0answers
59 views

Area Based Kolmogorov Smirnov

I have several sets to two ECDFs that I want to compare. I know I can use the Kolmogorov-Smirnov test (and I have) but one set of ECDF is returning statistically significant when I suspect the ECDFs ...
7
votes
0answers
2k views

Comparing two ECDFs using Kolmogorov-Smirnov test (alternative hypothesis)

I am taking measurements of a computer system performance over time and I'd like to understand if the performance is degrading or improving as time passes.. After doing some research, I picked the KS ...
1
vote
0answers
55 views

Evaluating model performance - different possible metrics

I am comparing the root mean squared error (RMSE) of different regression models that were fit to simulated data. I have RMSE from 30 different simulated data sets for each regression model. Some of ...
1
vote
2answers
275 views

Density of an empirical cdf

I can figure how the underlying density function of an empirical-cdf looks like? Does it look like a histogram?
4
votes
1answer
783 views

Estimating a cumulative distribution function from a mixture model

I have a problem I fail to find a solution. Assume a mixture model : $F(x)=a\times G(x)+(1-a)\times H(x)$ where $F$, $G$ and $H$ are cumulative distributions of different random variables, and $a$ ...
1
vote
1answer
579 views

Calculate probability of event from ecdf in R

I'm using R and would like to calculate the probability of an event happening. I used the ecdf function on daily revenue (month-to-date). I have a target and a daily number necessary to hit target ...
6
votes
0answers
335 views

Empirical distribution function of overlapping time series data

If we model asset return volatility for periods of more than one (say more than one day) there is the square-root rule which holds true under some assumptions. On the other hand practitioners ...
2
votes
0answers
42 views

Describing Differences between Two Inferred Populations

I am interested in the size of an animal in two populations, and found reliable estimates of the population provided by NOAA. The data was in the form of estimated total population within discrete ...
2
votes
1answer
1k views

How to compute joint cdf of an empirical copula? (Updated with more info)

lets suppose a bivariate empirical copula as: for a set of data of example data we can plot it like this: How can we compute the joint cdf of this empirical copula which should like this: Thank you
0
votes
1answer
227 views

When to use empirical cdf?

I saw in some papers that people sometimes use empirical cdf or complementary cdf to quantify the distribution of data. What is the advantage of using them and what interesting can they tell about ...
2
votes
2answers
762 views

What does a Barplot, a Boxplot and eCDF represent?

I want to understand what a barplot, boxplot, and eCDF represent when they are used for plotting data. A barplot is kind of intuitive, easy to understand what it represents. But I fail to understand ...
1
vote
1answer
1k views

Convergence in distribution with empirical distribution function (EDF)

I'm struggling with the following question: Let $F_n(x)$ denote the EDF of a random sample. Show that $\sqrt{n}(F_n(x)-F(x))\xrightarrow[]{d}N(0,F(x)(1-F(x)))$. I think that the course of ...
3
votes
1answer
203 views

Results stronger than Dvoretzky–Kiefer–Wolfowitz inequality?

There is the DKW inequality which controls the extent to which the empirical cdf of a sample from a real-valued random variable differs from the true cdf. Are there any stronger results (i.e. ...