Questions tagged [empirical-cumulative-distr-fn]

Empirical cumulative distribution function: a step function increasing by $1/n$ at each unique $X$-value that occurred in the sample.

Filter by
Sorted by
Tagged with
3 votes
1 answer
87 views

Finding Cumulative Distribution Function of weighted data

I have census tract data where each row holds the population size and a variable value (e.g., income). I want to plot the cumulative distribution function (CDF) of the TRUE population, i.e., ...
Ilik's user avatar
  • 133
0 votes
0 answers
36 views

Why not use eCDF to choose the model of survivability?

I have been reading a little bit about survivability and I am a little confused about choosing models. Let me set an experiment: You run 50 toasters until all of them fail, noting down their lifespan $...
Sorfosh's user avatar
  • 101
1 vote
0 answers
45 views

How do you test if two discrete ECDFs are drawn from the same population?

Background I have two Empirical Cumulative Distribution Functions (ECDFs) based on two samples of very different sizes. Sample 1: 1020 data points, Power-Law-like distribution, discrete data in the ...
Connor's user avatar
  • 615
0 votes
0 answers
18 views

Resample random variable to fit different variance

Suppose I have samples drawn from a random variable, and I want to multiply that random variable with a scalar constant. How should I transform the samples such that they would have been drawn from ...
mroelofs's user avatar
1 vote
0 answers
37 views

Proof of empirical expression of Cramer-Von Mises Statistics

I would like to know how to derive the empirical expression of the test statistics below: $$ \omega^2 = \sum_{i=1}^{n}(U_{(i)}-\frac{2i-1}{2n})^2 + \frac{1}{12n} $$ from $$ \omega^2 = n \int_\Omega(...
yoshitaka sasase's user avatar
1 vote
0 answers
58 views

Confidence interval for sum of product of scaled binomial random variables

I have discrete, independent, but not necessarily identically distributed random variables $X_1,\dots,X_n$ that take on non-negative integer values. Each random variable has unknown distribution ...
Efficiency's user avatar
0 votes
0 answers
144 views

Comparing the output distribution of two ML models

Consider a regression task (e.g. predicting house prices) with a given train and test sets. We start with constructing a linear regression model, in which we assume $y_i=X^T\beta+\epsilon$ with $E[\...
Spätzle's user avatar
  • 3,832
1 vote
1 answer
43 views

Transforming data with a fitted distribution function

I have a bivariate dataset on $[0,1]^2$ in which I am interested in fitting a joint distribution. I fit a Gaussian copula but am unsure how to judge if it's a good fit. I tried transforming my data ...
Bpe's user avatar
  • 13
1 vote
1 answer
55 views

Adjustment needed for multivariate Dvoretzky–Kiefer–Wolfowitz inequality on MCMC samples?

I was thinking about studying bounds on the multivariate empirical cumulative distribution function for samples from an MCMC chains. The multivariate Dvoretzky–Kiefer–Wolfowitz inequality would seem ...
Galen's user avatar
  • 8,412
2 votes
1 answer
98 views

Covariance between two binomial random variables or expectation of product of binomial random variables

I have an empirical distribution $S_n(x)$ (= proportion of samples less than equal to x) from a random sample $X_1, X_2, ..., X_n$ for a random variable $X \sim F_X$. Consider the random variable $T_n(...
Mewbacca's user avatar
4 votes
1 answer
173 views

Kolmogorov-Smirnov test with median ranks instead of traditional empirical distribution function

I know that Kolmogorov-Smirnov test uses the empirical distribution function of the sample studied $\widehat{F}(X_i) = \frac{i}{n}$ and then measures the adequacy of function $\widehat{F}$ to function ...
Martin's user avatar
  • 43
2 votes
1 answer
85 views

Convergence of histograms

Good morning, I'm interested in the convergence of empirical distribution functions. In particular, let us suppose to have a population with unknown distribution function $ F(x) $ and that I can ...
Lorenzo Eboli's user avatar
0 votes
0 answers
22 views

Random variables implicit in Glivenko-Cantelli theorem

Theorem 1 here states $\lim_{n\to\infty} \sup_x |\mathbb{F}_n(x) - F(x)| = 0$ with probability $1$, where $\mathbb{F}_n$ is the empirical distribution function of the first $n$ $X$s, which are i.i.d. ...
johnsmith's user avatar
  • 335
0 votes
1 answer
94 views

Compare CCDF of datasets with different sample size

I have 3 dataframes with the same structure (each dataframe includes a different type of tweet). Here are the columns of dataframes: id, ...
mOna's user avatar
  • 103
0 votes
0 answers
51 views

Estimation of Distribution using multiple ECDFs

Every day, I keep track of the processing times for each input to my CPU and create empirical cumulative distribution functions (ECDFs) based on this data. Let's assume I have 100 observations per day ...
smv's user avatar
  • 53
2 votes
2 answers
185 views

Quantize a continuous random variable

Suppose we have a continuous random variable $X$. We do not know its distribution function, but have $n$ i.i.d. samples. I am looking for methods that quantize (discretize) $X$ into a categorical ...
Mingzhou Liu's user avatar
1 vote
0 answers
16 views

How to apply a seasonal index into an eCDF?

I am doing a Before/After analysis, which is aiming at evaluating the effect of a change. Assume I have made a change at the beginning of June-2022, and I want to evaluate the effect of the change ...
Mostafa's user avatar
  • 111
0 votes
0 answers
80 views

Averaging ECDF vertically : Proof of convergence

Suppose we have a set A , we split into multiple disjoint subsets ai We only have access to the ai sets , is there a way to compute the ECDF for the set A without looking at it ? If for example we ...
Amine boujida's user avatar
0 votes
0 answers
64 views

Doubt in empirical distribution, sample and random variable

I have a basic doubt in sample and random variables. I have read related posts on this site but still some doubt is still left. Suppose we have a population and we are drawing some entries from it ...
Iti's user avatar
  • 115
0 votes
0 answers
86 views

How to compare two empirical CDFs obtained from current status data

I have failure time information of a product in the form of current status data, which looks something like this. Observed Time($t_i$) failed ? ($y_i$) 5 hrs 1 6 hrs 0 5 hrs 0 7 hrs 1 ...
aishik roy chaudhury's user avatar
0 votes
1 answer
138 views

How to plot a confidence band around the ecdf in the form (ecdf-b, ecdf+b) for a certain b in R?

I need to plot a confidence band around my ecdf. I calculated a value b and I basically just need to plot ecdf+b and ecdf-b but R doesnt let me do that. Does anyone know how this can be done?
Mina's user avatar
  • 1
0 votes
1 answer
73 views

Empirical distribution function by sampling from a m.v. distribution

I have mathematically rewritten my problem as a function of multiple iid variables: $$ f(X_1, X_2, ..., X_n), $$ where $$X_i \in \mathcal{N}(0,1)$$ I now want to determine the empirical distribution ...
Filip Johansson's user avatar
9 votes
1 answer
490 views

When was the earliest appearance of Empirical Cumulative Distribution Plots?

I would be surprised if we actually had a date here. I am curious who, if anyone, created the ecdf plot. When did the ecdf make its first appearance? If we do not know when the first ecdf plot was ...
Alex's user avatar
  • 2,011
5 votes
1 answer
98 views

Does this statistic comparing two samples using EDFs have a name?

So, I wanted to cook up a statistic that was similar to the Wasserstein metric for finite sized samples from distributions on a continuous support that is also invariant to reparameterization of the 1-...
Sean Lake's user avatar
  • 181
2 votes
2 answers
43 views

How to estimate correlations between the standard errors on empirical quantile estimates from two correlated series?

I have two very long series of samples from a pair of correlated random variables, $x$ and $y$. (they are fat tailed distributions, roughly Pareto, if that helps). (if you're curious...$x$ and $y$ are ...
Stephen Jewson's user avatar
6 votes
1 answer
307 views

Is there an equivalent to an ECDF with a "<" sign?

The empirical cumulative distribution function is defined as $$ F(x)=\frac{1}{n} \sum_{i=1}^{n} \mathbb I_{x_i \leq x} $$ Is there an equivalent interpretation of this function as: $$ F(x)=\frac{1}{n} ...
Tim Williams's user avatar
3 votes
1 answer
81 views

Covariance of the empirical probability mass function

Suppose a discrete random variable $Y$ takes $k$ levels of different values $y_1,y_2,...,y_k$. Let $P(Y=y_k):=p_k$. Suppose we have $n$ i.i.d. samples of $Y$, my question is: How can we compute the ...
Mingzhou Liu's user avatar
2 votes
0 answers
20 views

Simulating New Observations from a Empirical Distribution Function [duplicate]

Recently, I found out that it is possible to simulate new observations from an Empirical Distribution Function (EDF) . Suppose I collected some data (e.g. heights of baksetball players): $x_1, x_2,\...
stats_noob's user avatar
1 vote
0 answers
105 views

Empirical distribution for feature binning

In paper "A simple yet effective baseline for non-attributed graph classification" (https://arxiv.org/pdf/1811.03508.pdf) authors use empirical distribution for feature binning. Precisely, ...
qalis's user avatar
  • 239
2 votes
1 answer
141 views

symmetrization in glivenko-cantelli proof

In this proof of the Glivenko-Cantelli theorem, page 2 of these notes, two types of symmetrization are used. The first transforms the sup of the centered empirical cdf $$P(\sup_{z\in\mathbb{R}}|(1/n)\...
user19458's user avatar
1 vote
1 answer
53 views

rank to quantile estimate?

R> x=c(92, 3, 1, 4, 15, 4) R> rank(x) [1] 6.0 2.0 1.0 3.5 5.0 3.5 Given the rank results of an input vector of sampled data, I can estimate the quantiles ...
user1424739's user avatar
1 vote
1 answer
313 views

How to average several posteriors distributions from a Monte Carlo Simulation

Say you produce several posteriors distributions from different runs of the same model under different seeds. That is to say you have something like the following: ...
TTT's user avatar
  • 219
2 votes
1 answer
1k views

Distance between two categorical distributions

I want to test whether two (empirical) categorical distributions taking on $K$ possible values (e.g. 5, with no innate underlying ordering) with associated (empirical) probabilities $p_k$ are the same....
Dan.phi's user avatar
  • 67
0 votes
0 answers
153 views

Probabilities from ecdf and normalised data - is this acceptable difference? (using R)

I have a dataset of 500 observations, definitely not normally distributed: The minimum is around 50 000 in the data. If I use ecdf to predict e.g. 100 000, I get: ...
Reader 123's user avatar
2 votes
1 answer
660 views

What is a "bootstrap from the Empirical CDF"?

I am given the following set of questions. In all the questions the basic background is a sample X_1, ... , X_n. If n=3 and you observe the numbers 1, 2, and 4, what is the sample median. If you ...
Novice's user avatar
  • 571
0 votes
1 answer
618 views

Determine a product's rating given a known 3-sigma tolerance

Consider a 100Ω resistor with a 10% tolerance. We can assume this is the 3-sigma value since this is typical in manufacturing. Thus, we can expect ≈99.73% of such resistors to range from 90Ω to 110Ω. ...
IamIC's user avatar
  • 103
1 vote
1 answer
57 views

Difference in means

For a df that looks something like the following ...
freshman_2021's user avatar
0 votes
1 answer
144 views

Smoothed CDF to calculate asymptotic normality

If we have the following estimator: $\hat{F_Z}(z)=\frac{1}{N}\sum_{i=1}^N1\{Z_i\leq z\}$. The CDF of $Z$ is defined as $F_Z(z)=Pr(Z\leq z)$. $Z_1, ..., Z_N$ is i.i.d. data. What would be the steps to ...
Maybeline Lee's user avatar
2 votes
0 answers
135 views

Plug-in principle with kernel density estimate

The plug-in principle says that to estimate a statistical functional of the form $$ T(\mu) = \int f(x)\ d\mu(x) $$ we can replace $\mu$ with the empirical distribution $\mu_n$ depending on data $X_1,\...
TAlsup's user avatar
  • 196
7 votes
1 answer
270 views

What are the simplest examples of nonlinear statistical functionals?

I am reading Wasserman's book "All of Statistics" in which he defines a statistical functional as any function $T(F)$ of the cumulative distribution function $F(x)$ that outputs a real ...
Peaceful's user avatar
  • 603
2 votes
1 answer
61 views

Picking a specific estimated CDF from a set of CDFs provided by an ECDF

Let $F_X$ be a CDF of an unknown random variable $X$. If we have independent samples $x_1, x_2, \ldots, x_n$ of $X$ then we can estimate $F_X$ non-parametrically using an ECDF $\hat{F}_n$. By Central ...
euJue07's user avatar
  • 23
2 votes
2 answers
220 views

Notation for ECDF

I'm reviewing statistical functionals and U-statistics, trying to make notes, and I am tripping on notation. From my understanding, $X$ is used to denote a random variable and $x$ is used to denote an ...
eeeeejjjsss's user avatar
3 votes
1 answer
547 views

Density estimation from ECDF - numerical derivatives and scaled domains

Suppose we want to get a density estimate of some data X. One way is to compute the empirical CDF, ...
algae's user avatar
  • 53
1 vote
0 answers
31 views

Nonparameteric Empirical Estimator For Stochastic Process

Motivation: If $X$ is a random-variable defined on some probability space $(\Omega,\Sigma,\mathbb{P})$ then Glivenko-Cantelli lemma guarantees that the empirical distribution $\frac1{N}\sum_{n=1}^N \...
ABIM's user avatar
  • 544
0 votes
0 answers
172 views

How to deal with a PDF that takes CDF data as input? (Metalog distributions)

I am trying to utilise the Metalog distribution in a Machine Learning project. For this project, I need to be able to obtain likelihoods using the PDF of the distribution. https://en.wikipedia.org/...
Edrulesok's user avatar
  • 151
1 vote
0 answers
173 views

Is it possible to convert a pdf obtained with density() to an ecdf?

Consider the following code that gives us (an estimate of) the pdf of a random variable $X$: X = rnorm(100,10,1) XDensity = density(X) I want to obtain the ecdf of ...
sonicboom's user avatar
  • 920
2 votes
0 answers
301 views

Fitting a function to ECDF [closed]

I have some ECDFs. I would like to summaries the ECDFs with functional approximations. I was thinking that a polynomial, spline, or other line fitting procedure would generate a nice parsimonious ...
Alex's user avatar
  • 2,011
0 votes
0 answers
36 views

Computing the most representative sample of a random variable

Let $X$ be a real-valued random variable and $n > 0$. Using numerical methods, how can we find the vector $\vec v$ of $n$ real numbers that is most characteristic of $X$, in the sense that the ...
Kodiologist's user avatar
  • 20.1k
3 votes
1 answer
635 views

Median from two unmergable datasets

I am trying to calculate the "overall" median of a variable that is spread across two datasets. I have access to the raw data in each dataset but can't bring their raw data together. What ...
Arktik's user avatar
  • 133
3 votes
0 answers
1k views

Why are people using KS statistic to choose cutoff for binary predictions?

I've seen that people are sometimes using Kolmogorov-Smirnov statistic to determine a cutoff in a binary classification models, such as logistic regression. However, i do not fully understand why and ...
ira's user avatar
  • 451