Questions tagged [kolmogorov-smirnov-test]
The Kolmogorov-Smirnov test is a test for goodness of fit of data to a distribution. It is often used to test whether a variable is normally distributed.
612
questions
0
votes
1
answer
35
views
How to perform a 2 sample Kolmogorov-Smirnov test controlling for a type II error instead of type 1
I have four completely different sets of two samples and want to check for each set if both samples come from the same distribution, so four Kolmogorov-Smirnov tests total. If it's going to be ...
- 11
0
votes
0
answers
28
views
Test Distribution Difference between small sample sizes
My first post of StackExchange.
I am working on data analysis project for comparison of fraud and nonfraud transactions. My objective is to compare the feature vectors of nonfraud transactions and ...
0
votes
0
answers
31
views
testing difference in distributions with population values (using python)
I am looking for the best method to test for a difference between two distributions.
I have data from a patient group, and my aim is to compare the distribution to a reference distribution. However, ...
- 101
0
votes
0
answers
22
views
What Python library functions can one use to determine if a sample is drawn from a Laplace distribution?
I have some data I suspect is drawn from a Laplace distribution rather than a Gaussian one. I can use the Kolmogrov-Smirnov test and the ability to create Uniform and Gaussian distributions of the ...
0
votes
0
answers
65
views
What is this nonparametric goodness-of-fit test?
I wrote down a goodness-of-fit test that I have not seen before.
However, it is quite elementary and has many applications, so I
bet it must have been known. Could someone tell me its name?
Setup. The ...
- 235
0
votes
0
answers
30
views
Correctly interpreting a two-sample Kolmogorov-Smirnov test and a Kruskal-Wallis test
I want to confirm my understanding of the two tests referenced in the title.
Say I have data on a variable $Y$ from two groups (0 and 1), and I perform the following tests in Stata:
...
- 1
0
votes
0
answers
13
views
What is the relationship between convergence in measure and convergence in Kolmogorove distance?
Suppose I have a sequence of distribution $F_1, ..., F_n$ convergence to a limiting distribution $F$ in the sense that
\begin{align}
\sup ||F_n - F|| \overset{a.s.}{\rightarrow} 0
\end{align}
which ...
- 21
0
votes
1
answer
33
views
Tests that Quantify Deviation from Null Hypotheses
I have been delving into non-parametric tests recently, and I've come to realize that most of these tests offer only a partial perspective.
For example, lets say the underlying distribution is $\theta$...
- 235
1
vote
0
answers
35
views
Discrete P-values in Kolmogorov-Smirnov Test with Small Sample Sizes
I'm facing an issue while conducting the Kolmogorov-Smirnov (KS) test with two datasets of different sizes: 84 and 28 samples. I got these sets from normal distribution. I've noticed that the ...
0
votes
0
answers
34
views
How to test for significance of differences between metrics for two models? (Machine learning model selection)
Problem - I want to test whether the difference in a metric (say AUC) between two models is significant. I have one vector of binary class predictions from a custom function and one from sklearn....
- 1
4
votes
1
answer
147
views
Kolmogorov-Smirnov test with median ranks instead of traditional empirical distribution function
I know that Kolmogorov-Smirnov test uses the empirical distribution function of the sample studied $\widehat{F}(X_i) = \frac{i}{n}$ and then measures the adequacy of function $\widehat{F}$ to function ...
- 43
6
votes
1
answer
1k
views
Kolmogorov-Smirnov test fails to recognize a normally distributed sample
I created a sample with 10000 normally distributed numbers. Subsequently, I used the Kolmogorov-Smirnov test to check if they are indeed normally distributed, and it turned out that they are not. How ...
- 281
0
votes
0
answers
30
views
How to apply equivalence test of whether data come from a specified distribution?
I have evenly-spaced locations from $1, 2, \dotsc, 1000$. At each location I have data from my $N$ samples coming from two groups, and I apply a (one-sided) test to obtain a p-value. I want to know if ...
- 31
1
vote
1
answer
75
views
What test can I use to compare two peaks in an image?
This question is related but I want to check if my idea is valid.
I have two peaks and I want to check if they are the same. They are the linecut of an image. The background noise in the image is not ...
- 13
0
votes
0
answers
29
views
Relation between the sample size with the powers of Kolmogorov Smirnov, Cramer Von Mises and Anderson Darling?
I want to know the mathematical relation between the sample size with the powers of 1. Kolmogorov Smirnov, 2. Cramer Von Mises, and 3. Anderson Darling.
I have seen many posts and other websites ...
5
votes
2
answers
545
views
Kolmognorov Smirnov test p values is 0
I am trying to use Kolmogorov-Smirnov test to check the goodness of fit of the distributions for the dataset. I have dataset consisting of 100,000 samples and I apply expectation-maximization ...
- 117
2
votes
1
answer
76
views
How to determine whether a sample from a known population is significantly biased?
I have a large dataset (the population) and a large subset of it (the sample) containing the same, continuous variables. The sample represents more than 90% of the population but is not random -- we ...
- 133
2
votes
0
answers
30
views
Does the null distribution matter when choosing between goodness-of-fit tests?
There are multiple goodness of fit tests such as Kolmogorov-Smirnov, Cramer-von Mises, Anderson-Darling, and Kuiper's. Which one shall one use? There are threads (e.g. 1, 2, 3) and a nice textbook ...
- 64.7k
2
votes
1
answer
71
views
Comparison of KS test and MWW test
I want to compare two data samples which come from continuous distributions. However, the data is highly skewed and t-test is obviously not a choice. As an alternative, I wanted to use the MWW test ...
- 21
6
votes
1
answer
117
views
P-Value == 1 of two-sided Kolmogorov-Smirnov test for clearly distinct distributions
I have preformed two-sided Kolmogorov-Smirnov test pair-wise across 6 datasets. It performs as expected (p-value <<< 0.05 in most cases, ...
- 63
0
votes
1
answer
21
views
Is two-sample Kolmogorov test working correctly?
I have two one-dimensional samples that I'm trying to quantifiably distinguish (or deny such distinction). I.e. the null-hypothesis is that they come from the same population (distribution?). The ...
- 45
3
votes
1
answer
163
views
How does the assumption that the distribution must be fully specified when using a Kolmogorov-Smirnov practically impact comparison of distributions?
As specified here on the NIST website, the 3rd assumption for the K-S test is "... the distribution must be fully specified. That is, if location, scale, and shape parameters are estimated from ...
- 85
0
votes
0
answers
31
views
Two-sample Kolmogorov Smirnov test for global sensitivity analysis - How to treat discrete distributions?
Dear Cross Validated community,
We are working on a uncertainty & sensitivity analysis using a mathematical optimization model. More specifically, we have a set of uncertain parameters, which ...
- 101
0
votes
0
answers
37
views
Large data sets with heavy tails - best fitting strategy
I have relatively large data sets with about 20000 to 25000 data. The data are the orientation angles (0 - 180°) of fibers in a composite material. You can see two example histograms showing the most ...
- 1
1
vote
1
answer
47
views
Kolmogorov Smirnov Test Consistency
I was reading that the Kolmogorov Smirnov 2 sample test is consistent, that is Probability of rejection under $H_1$ is 1 for sample size going to infinity.
Say we have 2 random variables X and Y. K-S ...
- 37
0
votes
0
answers
38
views
Statistical tests for normality and log-normality testing?
I want to apply a filter to my data that assume its normality. The data comes from different sensors measuring the same quantity over the same period of time.
Using a simple histogram plot, It seems ...
- 13
0
votes
0
answers
17
views
Comparing ratings generated by overlapping groups of experts
Consider an experiment where randomly selected experts from a set are asked to attribute Likert-scale ratings to randomly selected objects from two groups A and B. Now, suppose I want to test whether ...
- 11
3
votes
1
answer
66
views
Are there any differences in R if I use a "pnorm" or create a distribution with rnorm for Kolmogorov-Smirnov?
For testing distributions existing in base R, say normal or beta distributions, is there any difference if I use a one-sampled or two-sampled Kolmogorov-Smirnov test with same parameters?
For example:
...
1
vote
1
answer
51
views
Upper bound on Kolmogorov-Smirnov distance after some transformation $h$
Problem setup: Suppose $X_1, \ldots, X_n$ is an i.i.d. sample from $F_X$ (CDF), and $Y_1, \ldots, Y_n$ is another i.i.d. sample from $F_Y$ (also CDF). In addition, $h(z_1, \ldots, z_n)$ is a real-...
- 33
2
votes
1
answer
104
views
Should we always use 100 samples for an equivalence test given the KS test size problems?
My problem:-
I'm writing a test to see if a series of numbers come from certain theoretical distributions. And I need a p value so that software can automatically accept or reject $H_0$ on an $\alpha=...
- 565
13
votes
2
answers
377
views
What are the advantages (if any) of the Kolmogorov-Smirnov test over other tests?
Given a reference distribution and an unknown sample, we need some statistical test to determine if the unknown sample came from the reference (one-sample test), or given two samples to determine ...
- 980
0
votes
0
answers
87
views
Comparing 3D distribution
I want to compare observations with a set of simulated data, but I'm struggling to find the best way of doing so:
My observations consist on a set of datapoints (~100) in a 3 dimensional space
I ...
0
votes
0
answers
21
views
How can I combine and get cdf of joint distribution of normal and beta distributions on the same set of data? [duplicate]
In my problem, I have a condition in which I need to compute the joint distribution of two dependent distributions. The first distribution is normal and the second one is beta distribution. How can I ...
- 31
0
votes
0
answers
34
views
Hypothesis testing for two samples from discrete distributions
There exists a classical method for solving a certain computational problem related to random sampling. It is the "gold standard", so to speak.
I'm working on an algorithm that aims to solve ...
- 107
8
votes
2
answers
928
views
Kolmogorov-Smirnov instability depending on whether values are small or big
Out of curiosity, I realised that the Kolmogorov-Smirnov normality test returns two very different p-values depending on whether the dataset has small or large numbers. Is this normal and is there a ...
- 169
0
votes
0
answers
66
views
How to simulate critical values for a two sample KS test?
I believe I managed to simulate critical values for a single sample with the following code in R.
...
- 11
1
vote
1
answer
201
views
How combine two statistics from different tests?
I have a case in which I have two sets of ranks. To compare these two ranks, I use once wilcox test and once ks test. The reason why I am using both returns to the special cases of shapes of data ...
- 31
1
vote
0
answers
33
views
Multiple Kolmogorov-Smirnov tests with likelihood ratio
I would like to be able to establish whether a sample (let's call it A) is more likely than another sample (let's call it B) to be drawn from the same distribution as a third sample (let's call it C).
...
- 111
0
votes
0
answers
23
views
Goodness-of-fit when independence assumption is violated
I would like to assess the goodness-of-fit of a theoretical model by comparing the actual distribution of values (the responses of subjects in an experiment) with the predicted distribution of values (...
- 21
4
votes
1
answer
194
views
Goodness-of-fit Tests
I wish to test whether a large number of observations $X_i$ follows an exponential distribution with parameter $\lambda=1$. I also wish to test this hypothesis exactly, and intend that if the ...
- 107
10
votes
2
answers
5k
views
Why are Shapiro-Wilk and Kolmogorov-Smirnov test compared with same sample size?
I am trying to understand when to use Shapiro-Wilk Test and/or Kolmogorov-Smirnov Test for normality. Currently, I am following this website How to Test for Normality in R (4 Methods). This paper ...
- 359
0
votes
0
answers
273
views
How do I test differences between two (non-normal) distributions?
I have a large dataset containing the peak velocities of different people. I have then split this dataset into two depending on an attribute (for example male vs. female). Each dataset now contains ...
2
votes
1
answer
127
views
Kolmogorov-Smirnov Test as a Measure of Statistical Distance Between Two Dependent Probability Distributions
Much has been asked on CV regarding the Kolmogorov-Smirnov (KS) test, but not in the context I need.
I'm wondering how feasible it would be to employ KS as a measure of distance between two dependent ...
- 1,537
0
votes
0
answers
66
views
Frequentists tests to check for normality
Let $X_1,...,X_n\sim X$ be $n$ i.i.d. random variables. I want to to test if they follow a normal distribution, in other words, check if their distribution belongs to the Gaussian family.
These are ...
- 107
0
votes
1
answer
46
views
Can we have different results for the same exemple using chi 2 and KS?
I just wanted to know, can we have different results for the same inputs, using Chi 2 and KS?
So for example I suppose that my variable flows a uniform law, as I do the test, the results I have are: ...
0
votes
0
answers
62
views
Test for uniformity of a dataset with many ties?
My goal is to identify images (2D matrix) which are approximately uniform (all same brightness).
Since I don't know of any "natural" tests for uniformity like Shapiro-Wilks for normality, I ...
- 101
0
votes
0
answers
87
views
Compare 2 binned non-parametric histograms with hypothesis test
According to this guy here: https://towardsdatascience.com/how-to-compare-two-distributions-in-practice-8c676904a285 and the following papers he mentions inside his article, and more specifically the ...
- 137
0
votes
0
answers
106
views
Chi Square Goodness of Fit or Kolmogorov–Smirnov test
I have a population binary data (2000 entries) as (pop_df):
...
- 135
1
vote
1
answer
491
views
How is Kolmogorov Smirnov a good measure of classifier performance?
KS tells how different two distributions are from each other. Assume I have a classifier model that has completely misunderstood the data and predicts a 0 for every 1 and a 1 for every 0. Assume the ...
- 111
0
votes
0
answers
167
views
Which non-parametric two-sample SciPy test to use for my data?
I have three datasets. They contain the angles between a set of vectors (V1) and three other sets of vectors (V2,V3,V5). The datasets should all be continuous and non-parametric, I don't expect a ...
