6
$\begingroup$

This is similar to the question Measuring the "distance" between two multivariate distributions, except that I want to measure distance between two data sets. I can imagine simply computing the mean, standard deviation, and other summary statistics and then somehow aggregating the differences for each distribution, but that does not seem very theoretically sound.

I wonder if there is a generally accepted method for doing this measurement.

$\endgroup$
  • 1
    $\begingroup$ duplicate of this question? $\endgroup$ – user603 Dec 3 '13 at 10:20
5
$\begingroup$

I think your question is essentially the same as Can I use Kolmogorov-Smirnov to compare two empirical distributions?, for which the Kolmogorov-Smirnov test is commonly used. The KS test statistic is a natural measure of the distance between two samples. For more details, please see Kolmogorov–Smirnov test on Wikipedia.

$\endgroup$
2
$\begingroup$

An area in statistics where this problem arises naturally is Approximate Bayesian Computation.

What you actually want to do is to summarise the whole sample into "informative" statistics that can later be compared using a suitable metric: this problem is not trivial at all. I would even say that it is actually one of the "hot topics" in statistics.

It is not that the method is not theoretical, it is just that there is not unique way of comparing two data sets. It usually depends on your aims and your model.

If you have a model and can identify sufficient statistics for it, then, by comparing the sufficient statistics of both samples you can assess how different the information contained in each sample is. If they are very close, then the associated [likelihood functions] (http://en.wikipedia.org/wiki/Likelihood_function) would be similar, and therefore the inferences on the corresponding parameters would be similar as well.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.