I want to test whether two discrete data sets come from the same distribution. A Kolmogorov-Smirnov test was suggested to me.
Conover (Practical Nonparametric Statistics, 3d) seems to say that the Kolmogorov-Smirnov Test can be used for this purpose, but its behavior is "conservative" with discrete distributions, and I'm not sure what that means here.
DavidR's comment on another question says "... You can still make a level α test based on the K-S statistic, but you'll have to find some other method to get the critical value, e.g. by simulation."
The version of ks.test() in the dgof R package (article, cran) adds some capabilities not present in the default version of ks.test() in the stats package. Among other things, dgof::ks.test includes this parameter:
simulate.p.value: a logical indicating whether to compute p-values by Monte Carlo simulation, for discrete goodness-of-fit tests only.
Is the purpose of simulate.p.value=T to accomplish what DavidR suggests?
Even if it is, I'm not sure whether I can really use dgof::ks.test for a two-sample test. It looks like it only provides a two-sample test for a continuous distribution:
If y is numeric, a two-sample test of the null hypothesis that x and y were drawn from the same continuous distribution is performed.
Alternatively, y can be a character string naming a continuous (cumulative) distribution function (or such a function), or an ecdf function (or object of class stepfun) giving a discrete distribution. In these cases, a one-sample test is carried out of the null that the distribution function which generated x is distribution y ....
(Background details: Strictly speaking, my underlying distributions are continuous, but the data tend to lie very near to a handful of points. Each point is the result of a simulation, and is a mean of 10 or 20 real numbers between -1 and 1. By the end of the simulation, those numbers are nearly always very close to .9 or -.9. Thus the means cluster around a few values, and I am treating them as discrete. The simulation is complex, and I have no reason to think that the data follow a well-known distribution.)