I am using the project jerzy to run a Two-sample Kolmogorov-Smirnov test in Javascript, regarding another question I asked on stats.SE: Timing attacks: When the time to complete two different tasks are statistically indistinguishable.
Not knowing anything about statistics, I would be grateful for some assistance interpreting the results. The code for the test being run is on Github (pieterprovoost/jerzy).
I am in particular running the test like this (and the numbers are contrived):
var diff = [750, 740, 790]; // array of nanosecond results
var equal = [750, 610, 960];
results.diff = new jerzy.Vector(diff);
results.equal = new jerzy.Vector(equal);
var ks = new jerzy.Nonparametric.kolmogorovSmirnov(diff, equal);
console.log(ks);
The output I get from the real data is something along the lines of:
{ d: 0.032657926102502954,
ks: 0.6660700343005954,
p: 0.7667168595417211 }
In the real tests the diff
and equal
are arrays of nanosecond timings. I would like to establish with some confidence that the arrays are effectively from the same distribution, with a difference of around 15ns.
How would one interpret the above result of the kolmogorovSmirnov
function of jerzy, in terms of how strongly one might state the probability and confidence that the two arrays are from the same distribution?