# two-sample Kolmogorov-Smirnov test suggest that sample with the same data generating process from different populations

I have two different samples I need to test if they are drawn from the same population distribution. The two samples are of the volatility of an asset that I simulated. Since the data is simulated I know that both samples have the same data generating process. The only difference between the two samples are the days that I am using from the simulation.

A summary of the two samples is given below. It is also important to note that since this is simulated data the sample size is quite large. I have 249,281 observations in sample 1 and 254,453 in sample 2.

> summary(Sample1)
Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
0.0128  0.0894  0.1194  0.1367  0.1627  0.9925
> summary(Sample2)
Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
0.0141  0.0950  0.1263  0.1467  0.1724  1.1435


When I apply the Kolmogorov-Smirnov test the results reject the null hypothesis that the two samples are drawn from the same population.

ks.test(Sample1,Sample2)

Two-sample Kolmogorov-Smirnov test

data:  Sample1 and Sample2
D = 0.053089, p-value < 2.2e-16
alternative hypothesis: two-sided


However if I plot the kernel approximation of the two different density functions and the CDF both of the samples appear to be from the same distribution. A copy of the plots is given below. How is it possible that the two samples come from different distributions if they have the same data generating process? Am I missing interpreting the results from the KS test? Is there another test that I should be apply?

• Try the simulation 1000 times. If you set $\alpha=0.05$, you should reject about $5\%$ of the time. If you're much different from that, then perhaps your simulation isn't as correct as you think.
– Dave
Commented Oct 7, 2020 at 22:07