Questions tagged [empirical-cumulative-distr-fn]

Empirical cumulative distribution function: a step function increasing by $1/n$ at each unique $X$-value that occurred in the sample.

Filter by
Sorted by
Tagged with
0
votes
0answers
8 views

Standard Errors of ECDF with Potential Model Fit Error

I have data and a proposed likelihood. I used MLE to fit the likelihood, but it is a very complicated likelihood and there is some estimation error of the model. The likelihood may be correct, but ...
6
votes
1answer
145 views

What are the simplest examples of nonlinear statistical functionals?

I am reading Wasserman's book "All of Statistics" in which he defines a statistical functional as any function $T(F)$ of the cumulative distribution function $F(x)$ that outputs a real ...
2
votes
1answer
23 views

Picking a specific estimated CDF from a set of CDFs provided by an ECDF

Let $F_X$ be a CDF of an unknown random variable $X$. If we have independent samples $x_1, x_2, \ldots, x_n$ of $X$ then we can estimate $F_X$ non-parametrically using an ECDF $\hat{F}_n$. By Central ...
2
votes
2answers
58 views

Notation for ECDF

I'm reviewing statistical functionals and U-statistics, trying to make notes, and I am tripping on notation. From my understanding, $X$ is used to denote a random variable and $x$ is used to denote an ...
0
votes
0answers
14 views

Trying to interpret the comparison between two ECDF curves

I am trying to understand ECDFs to compare two models "a" and "b". My initial interpretation from the plot is that judging by the difference between the two curves model "b&...
2
votes
1answer
119 views

Density estimation from ECDF - numerical derivatives and scaled domains

Suppose we want to get a density estimate of some data X. One way is to compute the empirical CDF, ...
0
votes
0answers
19 views

Bootstrap bias-corrected percentiles

Following [1] p. 185-186 the $BC_a$ (bootstrap corrected and accelerated) confidence intervals are given by: $$ (\hat{\theta}^{*(\alpha_1)},\hat{\theta}^{*(\alpha_2)}), $$ where $\hat{\theta}^{*(\...
2
votes
0answers
21 views

Nonparameteric Empirical Estimator For Stochastic Process

Motivation: If $X$ is a random-variable defined on some probability space $(\Omega,\Sigma,\mathbb{P})$ then Glivenko-Cantelli lemma guarantees that the empirical distribution $\frac1{N}\sum_{n=1}^N \...
0
votes
0answers
20 views

Kolmogorov-Smirnov (KS) test two-sample eCDFs and Dmax

I am running the two sample KS test to see whether two models are similar or not. Now the KS test does give me the answer to my question. But I have a question regarding the eCDFs and Dmax computed by ...
0
votes
0answers
7 views

Understanding of scale loss with Gaussian prior

I am reading a paper called PHOSA wherein a scale loss is used. I had the following questions regarding this scale loss in equation 8: What part of this loss function is actually incorporating a ...
0
votes
0answers
26 views

How to deal with a PDF that takes CDF data as input? (Metalog distributions)

I am trying to utilise the Metalog distribution in a Machine Learning project. For this project, I need to be able to obtain likelihoods using the PDF of the distribution. https://en.wikipedia.org/...
0
votes
0answers
18 views

Convergence of the first moment of empirical distribution

Imagine I have a sequence of random variables $\{x_n\}_{n=1}^{\infty}$. These are not i.i.d. random variables, but an arbitrary sequence. For any $n$, I can define the empirical CDF function $F_n(t) = ...
0
votes
0answers
13 views

Reverse Cumulative Frequency explanation

I've been seeing a lot of Reverse Cumulative Frequency (RCMF) plots lately and trying to figure out how to understand the way reverse cumulative frequencies are calculated, and the interpretation of ...
1
vote
0answers
32 views

Is it possible to convert a pdf obtained with density() to an ecdf?

Consider the following code that gives us (an estimate of) the pdf of a random variable $X$: X = rnorm(100,10,1) XDensity = density(X) I want to obtain the ecdf of ...
2
votes
0answers
38 views

Fitting a function to ECDF [closed]

I have some ECDFs. I would like to summaries the ECDFs with functional approximations. I was thinking that a polynomial, spline, or other line fitting procedure would generate a nice parsimonious ...
0
votes
0answers
19 views

Computing the most representative sample of a random variable

Let $X$ be a real-valued random variable and $n > 0$. Using numerical methods, how can we find the vector $\vec v$ of $n$ real numbers that is most characteristic of $X$, in the sense that the ...
3
votes
1answer
57 views

Median from two unmergable datasets

I am trying to calculate the "overall" median of a variable that is spread across two datasets. I have access to the raw data in each dataset but can't bring their raw data together. What ...
1
vote
0answers
69 views

Why are people using KS statistic to choose cutoff for binary predictions?

I've seen that people are sometimes using Kolmogorov-Smirnov statistic to determine a cutoff in a binary classification models, such as logistic regression. However, i do not fully understand why and ...
3
votes
1answer
38 views

Obtaining an expression for empirical mean from empirical CDF definition

This is my first post so I will try to be as clear and concise as possible. I am doing a course in statistics and we define the true mean of a random gaussian variable to be as follows: $\mu$ = $\int_{...
0
votes
1answer
137 views

Confidence Interval of p-Quantile from Empirical CDF

I am trying to provide an interval estimate for the 0.8-quantile of some numeric data, which is assumed to be an IID sample from some unknown, continuous distribution. I constructed an Empirical CDF ...
0
votes
0answers
45 views

Calculating the quantile of random forest test cases

I want to calculate the quantile of the observed value of a test case with respect to the prediction interval generated from a random forest, so for each test case I want the proportion of the ...
0
votes
0answers
21 views

Goodness of fit methods for density estimation

If we want to estimate the probability distribution function (pdf) of finite-sampled real continuous data using one of the following approaches: Parametric density estimation: fit a well-known ...
0
votes
0answers
40 views

Empirical estimation of conditional distribution $Y|X$ at the boundaries of X

I want to estimate conditional distributions of Y | X. Where X contains several continuous covariates. I'm coding in R. I tried several methods so far, but none gives me entirely satisfactory results ...
1
vote
1answer
50 views

Does the Glivenko-Cantelli theorem work back and forth?

If we have a sample $X_1, X_2, \ldots, X_n \sim F$ then $\hspace{1mm}sup_x|F_n(x) -F(x)|\xrightarrow{a.s./p}0$. Now, if I can come up with a theoretical cdf $F$ such that $\hspace{1mm}sup_x|F_n(x) -F(...
0
votes
0answers
91 views

Is it accurate to take the maximum distance between CDF and ECDF only at the edges? (Kolmogorov-Smirnov Test)

I have two samples, one obtained empirically and the other is the result of a simulation. I want to tune the simulation so that the result resembles the reality, for that I will minimize the KS ...
0
votes
1answer
94 views

Example of cumulative distribution function and the empirical distribution function [closed]

A random of 100 rolls of the die. The outcomes 1, 2, 3, 4, 5, 6, occurred 13, 19, 10, 17, 14, 27 times, respectively. Calculate the cumulative distribution function and the empirical distribution ...
0
votes
1answer
188 views

How can i find empirical survival function using survival function in R?

The survival function is given by: S(y; α, λ) = (α/α−1)* (1 − α^(−e^(−λy ))), if α is not equal 1 = e^(−λy) if α =1 y = 1 4 4 7 11 13 15 15 17 18 19 19 20 20 ...
37
votes
4answers
2k views

Intuitive explanation of Kolmogorov Smirnov Test

What is the cleanest, easiest way to explain someone the concept of Kolmogorov Smirnov Test? What does it intuitively mean? It's a concept that I have difficulty in articulating - especially when ...
0
votes
0answers
26 views

Predicting probability of failure mid life with weibull or ecdf

thank you for taking a look at this. I have failure data for tires over a 5 year period. For each tire, I have the start date(day0), the end date(dayn), and the number of miles driven for each day. I ...
3
votes
0answers
142 views

Comparing empirical distributions with an interaction effect

My experiment includes two subject groups: group1 and group2. Both groups undergo a behavioral test, the result of which is called “movement celerity” (this is my dependent variable). Both groups are ...
0
votes
1answer
31 views

Fit cdf and pdf from points on empirical cdf

I have cumulative counts with respect to a variable x, which looks like: ...
1
vote
1answer
51 views

Can we apply the Probability Integral Transform to Dependent Random Variables?

Let's suppose we deal with a non-homogeneous Poisson process having intensity function λ(t), t ≥ 0. The event times X1, X2, … of ...
0
votes
1answer
42 views

How to perform van Zyl's (2018) ecf-based normality test in R [closed]

I have been reading quite a bit on empirical characteristic function (ecf) based normality tests, but cannot find any functions to perform such a test in R and lack the mathematical ability to figure ...
2
votes
0answers
34 views

What are “absolute ECDFs” called (if anybody uses them)?

By absolute ECDF, I am referring to an ECDF that shows the absolute number of samples above some value as opposed to the fraction of such samples. For example, the bottom plot below displays such an ...
1
vote
0answers
20 views

Quantile-calculation example: Why the term $c$?

I need help to interpret a solution to the following example: Based on historical claim amounts $x_1,...,x_{47}$, which are assumed to be outcomes from an unknown claim distribution, you are ...
0
votes
2answers
292 views

Simulate data from ECDF in R [closed]

Consider the following 11 continuous non-negative observations: 0, 0.3, 0.31, 0.33, 0.37, 0.49, 0.51, 0.53, 0.59, 0.6 Obtain the empirical cumulative distribution function for these observations. ...
2
votes
1answer
70 views

How to quantify distribution concentration in cdf? [closed]

For example, let's say I have a list of users, each user has its revenue. I can plot the cdf of both user and revenue, to see if there is some concentration, for example, may be 40% user contribute ...
2
votes
0answers
88 views

Measuring distance between cumulative distribution and empirical distribution

What is an easy to understand step by step procedure on how to compute a distance between a cumulative distribution function and an empirical distribution function given a random sample using ...
8
votes
3answers
993 views

Confidence Interval of CDF

I am trying to determine if there is a statistically meaningful distinction between the cumulative probability density curves shown in the figure below. It's simple enough to do a $t$-test on the ...
0
votes
0answers
30 views

How can I identify an unfamiliar cumulative distribution function?

I have 116 Bessel-corrected sample variances (average of squared distances from sample mean), each from a sample of three measurements. All measurements were done using the same method. I had ...
2
votes
0answers
38 views

What linear transformation minimizes the Kolmogorov–Smirnov distance between two sets of points?

Suppose we have two sets of real-valued points $X_1, \dots, X_m$ and $Y_1, \dots, Y_n$. We want to find the linear transformation that makes the distributions look alike. If we want them to have the ...
4
votes
1answer
330 views

MLE as an expectation over the empirical distribution

I am reading Ian Goodfellow "Deep Learning" book. At page 128, it writes the maximum log-likelihood estimator and then says it is equivalent to the expectation over the empirical distribution To ...
5
votes
1answer
251 views

Maximum likelihood as minimizing the dissimilarity between the empirical distriution and the model distribution

I am reading Ian Goodfellow "Deep Learning" book. At page 128 it says One way to interpret maximum likelihood estimation is to view it as minimizing the dissimilarity between the empirical ...
0
votes
1answer
36 views

Determine significance of an observed value

I have an ECDF of values that do not follow a particular distribution (thought they are slightly normal, they are not). And I wish to determine if a new observed value is significant or an outlier or ...
1
vote
0answers
46 views

Empirical 1-sigma confidence interval

I would like to construct a 1-sigma confidence interval for my 1D data. I don't know the underlying distribution, and it is strongly skewed, so standard deviation will not suffice. I see that people ...
2
votes
2answers
278 views

Fitting a normal CDF using proportion data

I have the following data (prop is like empirical CDF): ...
3
votes
2answers
290 views

Can some one give a step-by-step computation for an empirical distribution with this concrete example

This post gives this formula to compute the empirical distribution: \begin{equation} \hat{p}(x) = \frac{1}{m} \sum_{i=1}^m \delta(x - x^{(i)}) \tag{3.25} \end{equation} I am trying to compute this ...
0
votes
1answer
1k views

obtaining empirical CDF of a given data

I have a dataset of variable $x$ that has a value between 0 and 6. I would like to have a function that defines empirical CDF of variable $x$. Since $x$ does not have a specific distribution (such as ...
0
votes
1answer
248 views

Is the Cumulative Hazard Function simply estimated using the Empirical CDF?

I'm just starting to learn about Survival Analysis and I understand the theoretical proofs but I'm still unclear about how to do some of the estimations practically.
2
votes
0answers
93 views

R: Calculating the convolution of two (multivariate) functions using FFT

I'm looking for a way to calculate: $$(f\ast g)(x) = \int_{\mathbb{R}^d}f(y)g(x-y)dy$$ in R. I have solved this problem using Monte-Carlo integration. However, ...