2 added 225 characters in body edited Dec 3 '13 at 10:41 AlphaOmega 2922 bronze badges An area in statistics where this problem arises naturally is Approximate Bayesian Computation. What you actually want to do is to summarise the whole sample into "informative" statistics that can later be compared using a suitable metric: this problem is not trivial at all. I would even say that it is actually one of the "hot topics" in statistics. It is not that the method is not theoretical, it is just that there is not unique way of comparing two data sets. It usually depends on your aims and your model. If you have a model and can identify sufficient statistics for it, then, by comparing the sufficient statistics of both samples you can assess how different the information contained in each sample is. If they are very close, then the associated [likelihood functions] (http://en.wikipedia.org/wiki/Likelihood_function) would be similar, and therefore the inferences on the corresponding parameters would be similar as well. An area in statistics where this problem arises naturally is Approximate Bayesian Computation. What you actually want to do is to summarise the whole sample into "informative" statistics that can later be compared using a suitable metric: this problem is not trivial at all. I would even say that it is actually one of the "hot topics" in statistics. It is not that the method is not theoretical, it is just that there is not unique way of comparing two data sets. It usually depends on your aims and your model. If you have a model and can identify sufficient statistics, then, by comparing the sufficient statistics of both samples you can assess how different the information contained in each sample is. An area in statistics where this problem arises naturally is Approximate Bayesian Computation. What you actually want to do is to summarise the whole sample into "informative" statistics that can later be compared using a suitable metric: this problem is not trivial at all. I would even say that it is actually one of the "hot topics" in statistics. It is not that the method is not theoretical, it is just that there is not unique way of comparing two data sets. It usually depends on your aims and your model. If you have a model and can identify sufficient statistics for it, then, by comparing the sufficient statistics of both samples you can assess how different the information contained in each sample is. If they are very close, then the associated [likelihood functions] (http://en.wikipedia.org/wiki/Likelihood_function) would be similar, and therefore the inferences on the corresponding parameters would be similar as well. 1 answered Dec 3 '13 at 10:34 AlphaOmega 2922 bronze badges An area in statistics where this problem arises naturally is Approximate Bayesian Computation. What you actually want to do is to summarise the whole sample into "informative" statistics that can later be compared using a suitable metric: this problem is not trivial at all. I would even say that it is actually one of the "hot topics" in statistics. It is not that the method is not theoretical, it is just that there is not unique way of comparing two data sets. It usually depends on your aims and your model. If you have a model and can identify sufficient statistics, then, by comparing the sufficient statistics of both samples you can assess how different the information contained in each sample is.