# Use of two-sample Kolmogorov-Smirnov test to evaluate similarities between two different distributions

$X_1, X_2, \dots X_n$ and $Y_1, Y_2, \dots Y_n; n = 1000$ are two samples of physical quantities coming from the application of two different mathematical models to some independent and identically distributed (iid) data.

The mathematical model used to generate $Y_i%$ is a simplified version (tuned by the parameter $r$) of the mathematical model used to generate $X_i$.

My goal is to find the value of $r$ that makes the empirical distribution $F_Y(y)$ to be as similar as possible to the distribution $F_X(x)$.

I decided to use the Two-Sample Kolmogorov-Smirnov test (in R) for different value of $r$ ranging in a specific interval. Is this choice correct?

I know that the null hypothesis for the K-S test is that the two distributions are the same. However, I know for sure that the two distributions are different because the two mathematical models are different. Is it correct to evaluate the best value of $r$ by looking at the p-value and the D statistic coming from the K-S test?

• Welcome to the site, @RiccardoCavallari. Please don't sign your posts. Notice that your username (with a link to your user page) & your flair are automatically added to your posts. Regarding your question, do you care if the mean or SD is similar, or only if the shape & ranking is similar? Could you just correlate the 2? Commented May 13, 2013 at 14:01
• Thank you @gung, actually, since the two distributions are surely different, it could be enough to find the value of the parameter r that makes the mean and the standard deviation similar. If this is the case, what are the best tools to use? Commented May 13, 2013 at 14:31

You say you're trying to make the two distributions close; the $D$ statistic measures the discrepancy between the two, and (as long as it's not changing the dimension of the fit) choosing $r$ to minimize that discrepancy makes complete sense.
I don't think there's any need to deal with the $p$-value; $D$ is a sensible thing to optimize.
If you're changing the dimension of the fit (adding or removing parameters), just minimizing $D$ isn't going to be sufficient, since more parameters will always tend to improve the fit.
• It's not clear to me what information you might need. If changing $r$ does something like change the amount of smoothing, you may change the dimension of the model with $r$. Are you able to describe what $r$ does? Commented May 13, 2013 at 14:26
• But if the $p$-value is large, does it matter if $D$ is small w.r.t. establishing similarity? Commented Dec 9, 2013 at 0:31