Questions tagged [kolmogorov-smirnov-test]

The Kolmogorov-Smirnov test is a test for goodness of fit of data to a distribution. It is often used to test whether a variable is normally distributed.

Filter by
Sorted by
Tagged with
1 vote
1 answer
35 views

How is Kolmogorov Smirnov a good measure of classifier performance?

KS tells how different two distributions are from each other. Assume I have a classifier model that has completely misunderstood the data and predicts a 0 for every 1 and a 1 for every 0. Assume the ...
  • 111
0 votes
0 answers
28 views

Which non-parametric two-sample SciPy test to use for my data?

I have three datasets. They contain the angles between a set of vectors (V1) and three other sets of vectors (V2,V3,V5). The datasets should all be continuous and non-parametric, I don't expect a ...
1 vote
0 answers
47 views

Comparing two distributions using t-test and confidence intervalls

I have two distributions (see histograms below) and I would like to know, whether they are significantly different (let's say alpha=0.05). The sample size is n=500 in both cases (obtained by ...
  • 111
0 votes
0 answers
18 views

KS Test with Replicate Experimental Observations

I am planning to run 2-sample KS test on experimental data I have to determine whether some specific subsets of the observations have a different distribution from the set as a whole. The signals I am ...
4 votes
1 answer
74 views

Is it possible to make a confidence envelope for a two sample Q-Q plot in R (or Python)? If so, what is the simplest method?

I want to show the confidence envelope for a two sample Q-Q plot in R (or Python). The aim is to use the Q-Q plot to give an indication of whether my two samples are drawn from the same population The ...
0 votes
0 answers
23 views

What does Kolmogov-Smirnov tells us when the p value is high?

I have a dataset which seems to be uniform by eye-examination. I want to verify if this distribution is indeed uniform. I saw a couple of questions in here which suggested using a Kolmogorov-Smirnov ...
3 votes
1 answer
35 views

Why is p-value produced by the K-S test decreasing with increased sample size even though samples are from the same distribution?

I have two samples that I know a priori were drawn from the same distribution. When I increase the size of the samples, the K-S statistic dips significantly (expected), but so does the p-value, ...
0 votes
0 answers
15 views

why Goodness of fit tests for PDF gives 0 p-value for large data sets? [duplicate]

I am working on traffic data, consisitng of data points in ten of thousands.I run goodness of fit tests (K-S,Anderson-Darling,Chi-squared) to check which Probabilty distribution model is best fit for ...
0 votes
0 answers
60 views

Can i use Kolmogorov-Smirnov on categorical variables

i have two datasets A and B i use the KS statistic to understand if A has come from the B Now, i have the variable c which is categorical. It takes values between 400 to 744. Each number shows a type ...
  • 335
0 votes
0 answers
23 views

Why do we need a binary tree for this way of computing the two-sample Kolmogorov-Smirnov statistic?

In this paper, Detecting Change in Data Streams, section 5 Algorithms, the authors show a way to compute the two-sample Kolmogorov-Smirnov statistic over intervals and initial segments in $O(log(m_{1}+...
  • 155
0 votes
0 answers
34 views

Is there any post hoc test after 2- sample Kolmogorov–Smirnov test?

I have 6 distribution as a result of 6 different simulation experiments. I performed 2-sample Kolmogorov–Smirnov test to check whether the 6 distributions are different from a default distribution ...
  • 1
4 votes
1 answer
169 views

Usefulness of KS tests and other similar distribution comparing tests

I am working on a machine learning binary classification problem. I have an outcome variable status called as loan paid and <...
  • 2,096
1 vote
1 answer
46 views

Manually estimate KS statistic

I use the proc npar1way with the edf option to produce the KS statistic. My aim is to use KS to understand if the sample (smp) is different from the population (pop). When I run the proc npar1way I ...
  • 335
0 votes
0 answers
195 views

Gini Coefficient and KS statistic conflict

What would it mean if the interpretation of the gini coefficient and KS statistic conflict with one another? In a scorecard model, if they were to conflict should you rely on one over the other?
  • 1
0 votes
0 answers
14 views

Finding an optimal split of a sample, based on distance of sub-sample's empirical distributions

Consider a sample $X$, consisting of $n$ observations that are independent (but not necessarily identically distributed), as well as two splits $S_1 := (X_{1,\dots,i}, X_{i+1, \dots, n})$ and $S_2 := (...
0 votes
0 answers
73 views

KS-Test and KL-divergence have diffrent result

It is a similar question to this but it didn't help me When KL Divergence and KS test will show inconsistent results? I have run into a situation in which I have no clue how to interpret it. I tried ...
  • 83
2 votes
0 answers
43 views

Test for distribution equality

This question touches Kolmogorov-Smirnov testing, but asks actually something different. Consider independent random variables $X_1, \dots, X_n$. I want to test the following hypothesis: $$ H_0: X_i\ ...
0 votes
0 answers
21 views

Kolmogorov-Smirnov inconsistency with increasing dataset sizes

I want to compare two arbitrary distributions and know that Kolmogorov-Smirnov is one such test that can handle any non-normal distribution. I noticed however, that p-values returned by the KS test ...
  • 101
8 votes
2 answers
2k views

Kolmogorov-Smirnov Test in Python weird result and interpretation

I have difficulties to understand how the Kolmogorov-Smirnov Test works. If I want to know if my samples are from a specific distribution (for example from the weibull distribution) I can compare my ...
  • 83
0 votes
0 answers
11 views

Detecting Mean differences on streaming data

Im taking online measurements on highly skewed real valued data of two possibly unrelated distributions. The precision is low, which means small values tend to come in duplicates, while large values ...
0 votes
0 answers
48 views

How to compare distributions with errors on the data points?

Here's a mock set-up of my problem: I have two non-normal probability density distributions (PDFs), $A$ and $B$. Distribution $A$ has error measurements for each data point while distribution $B$ ...
1 vote
2 answers
281 views

Kolmogorov-Smirnov p-value and alpha value in python

I am having trouble understanding the scipy.stats documentation against the backdrop of a book that explains the KS-Test. The documentation (https://docs.scipy.org/doc/scipy/reference/generated/scipy....
2 votes
0 answers
25 views

Correct hypothesis test to use here?

I have 5 samples from which I gathered data at 5 separate time points (days). Each day I observe the distribution of some RV. When I plot the distribution for each day it sort of looks like this for ...
  • 21
1 vote
1 answer
44 views

Significance level to accept null hypothesis using KS Test

Apologies in advance if I am thinking about this in the totally wrong way. Essentially, I have a theoretical distribution that I want to test, using the KS test, if my numerical data follows the same ...
  • 123
1 vote
0 answers
94 views

KS test for more than two variables

If we have two random variables $X,Y$ and their random sample $X_1, \dots, X_n, Y_1, \dots, Y_m$. We can test whether $F_Y=F_X$ by classical Kolmogorov Smirnov test, or AD test etc ($F_X$ is the ...
0 votes
0 answers
20 views

Time Series with Possible Discontinuity - Is the discontinuity significant?

I am just beginning to learn about statistics and statistical tests, so please forgive me if I'm using entirely the wrong terminology or I've missed something obvious. I'm trying to solve a problem ...
2 votes
0 answers
236 views

KS-statistic decile seperation - significance?

I already referred the posts here and here but they are different and doesn't answer my problem. I am currently working on a binary classification problem with random forests, neural networks etc. ...
  • 2,096
0 votes
0 answers
20 views

Same distribution failed and passed Kolmogorow-Smirnow-Test before and after normalization? [duplicate]

Below is the python code for the KS test. ...
1 vote
0 answers
17 views

Formula for Leemis's m

I'm looking for the formula to calculate the statistics and the significance level of Leemis's m. Wikipedia reports the statistics as $$m = \sqrt{N}\cdot\max_{i=1}^9 \left\{ \left|\Pr (X \text{ has ...
  • 243
1 vote
0 answers
44 views

Can I use both a Kolmogorov–Smirnov test and a T-test to compare two distributions?

I have two distributions (N ~ 300, μ = 1.89 and σ = 0.97) and (N ~ 40, μ = 1.74 and σ = 1.18). A ks_2samp returns statistic = 0.2 and p-value = 0.057. Therefore, I cannot reject the null hypothesis ...
1 vote
0 answers
23 views

Unclear as to purpose of K-S test & U test in this study

I'm currently looking at this study's statistical measures: https://www.sciencedirect.com/science/article/pii/S0191886920302531?via%3Dihub Somewhere in the paper I noticed this: I have some questions ...
9 votes
3 answers
1k views

Is it reasonable to use the Kolmogorov-Smirnov test to assess the normality of a random variable?

Context I am confused by the following post where the accepted answer states that : You can't really even compare the two since the Kolmogorov-Smirnov is for a completely specified distribution (so ...
1 vote
2 answers
155 views

Kolmogorov–Smirnov statistic/distance is normally distributed?

Suppose we have two populations X and Y, then compute the KS statistic/distance for samples drawn from them. Is this KS statistic/distance normally distributed? If so, how to prove it? If not, what ...
  • 867
2 votes
1 answer
59 views

Finding percentiles without complete data set

I have a set of incomplete data for wealth of a given population. Whilst I have the wealth of these individuals (around 27% of the population), I do not have the wealth of every individual. I do have ...
  • 25
1 vote
2 answers
140 views

Convergence in distribution and convergence in Kolmogorov distance

Let $X, Y$ be two random variables with laws $F$ and $G$ respectively. The Kolmogorov distance between these two laws is defined as: $$ d_{Kol}(F, G) = \sup_{x \in \mathbb R} |\mathbb P(X \leq x) - \...
  • 23
0 votes
0 answers
160 views

How to Compare Data Distribution of 2 large datasets?

I want to compare two super large datasets (petabytes). I have tried to use scipy ks_2samp kolmogoroc smirnov 2 sample test, but I get all the pvalues to be 0. I tried to downsample by random sampling ...
  • 21
2 votes
1 answer
458 views

Kolmogorov–Smirnov statistics as distance

I have a feature measured for each institution in each city. I want to calculate similarity between cities using the institutions' features. I think that if I compare the distributions pairwise and ...
  • 123
1 vote
1 answer
47 views

Test and minimum sample size to conclude that 2 ECDFs are different

Let's say I have samples $X$ from one distribution and I am now collecting new samples $Y$ from another unknown distribution. What would be the best test to check whether the samples $Y$ come from the ...
  • 119
0 votes
0 answers
13 views

How do I make statements about parts of distributions?

Let the following be the situation: I am measuring the occurrences of certain events (say "smiles") of individuals in the two situations A and ¬A, from which I calculate an index between -1 ...
  • 101
0 votes
0 answers
121 views

KS-Test on Discrete Distributions on Poisson-type Process

I have two distributions of data from a Poisson-like process. The data is based on randomized searches and tallies the amount of results found in one hour. One distribution has over 300 searches, ...
  • 123
0 votes
0 answers
59 views

KS test or chi square for comparing two distributions of a discrete ratio variable

I have a discrete ratio variable (length) from two samples and I'd like to know if the distributions are different or the same. In my case length is discrete because it can only take on integer values ...
0 votes
0 answers
282 views

KS test on exponential distribution

I'm trying to determine how well an observed distribution matches a theoretical one However, if I do a KS test as I've been taught on this, the highest difference (KS statistic) between the two ...
  • 101
1 vote
1 answer
59 views

Pass KS Test Fail Z Score Test

Im working with some messy data and was comparing two data sets. I used both the KS test and the Z-Score I found instances where Means would be found to be different under the Z score test but the ...
0 votes
0 answers
25 views

Kolmogorov-smirnov test to compare how similar different pairs of samples are

Say I have a list of samples in pairs: [(A, B), (C,D), (E, F)], where A,B,C,D,E,F are different samples and can be of different sizes. Now if I want to compare which of these pairs are statistically ...
  • 101
1 vote
1 answer
474 views

Wasserstein distance and Kolmogorov-Smirnov statistic as measures of effect size

So I've been dealing with 2 sample hypothesis testing with very large samples (around 20,000s each). Whenever I test for the equality of distribution I always reject the null hypothesis, even though ...
  • 23
1 vote
2 answers
51 views

What is the theoretical justification of testing for equal mean/distribution via resampling?

If we have two data sets $X_1,\ldots,X_m$ and $Y_1,\ldots,Y_n$, each i.i.d., and wanted to determine whether $\mathbb{E}[X_1] = \mathbb{E}[Y_1]$ or not using $\bar{X}_m - \bar{Y}_n = \hat{\Delta}_{m,n}...
  • 1,143
0 votes
0 answers
28 views

What is the different between 2 codes for K-S test

What is different between these two codes and when can we use them I use K-S test code in R to check normaliity for data that is generated from N(1,6) of size 5 ...
0 votes
0 answers
31 views

Aggregating individual ks 2 samp results?

Lets say I have two servers each reporting an individual time series of cpu usage, ts_a and ts_b. I can average these two CPU usage metrics together to get an averaged or "composite" time ...
0 votes
0 answers
69 views

K-S test for weighted histogram/samples

I have the following problem: I am estimating two PDFs using histograms, and I would like to assess how different the two are. This is in principle a classical example for a K-S test. However, I need ...
0 votes
0 answers
108 views

Why does R give different value for one-sample K-S test?

I want to do K-S test in R. In this case, I use AdequacyModel and ks.test in R. My data: ...
  • 41

1
2 3 4 5
12