2
$\begingroup$

I have two multivariate data sets, one is simulated, and I want a test to compare how similar their distributions are. The data set is discrete and the distribution is unknown. In other words, I am looking for an alternative for KS, AD or similar univariate statistical tests that cab be applied for multivariate data. I appreciate your answers.

$\endgroup$
  • $\begingroup$ There are innumerable ways that distributions can differ, even within univariate distributions. This will be even more so w/ multivariate distributions. No test could assess all possible ways 2 distributions could differ. So the key question is, what kinds of differences are you most concerned about? $\endgroup$ – gung - Reinstate Monica Feb 27 '17 at 1:13
2
$\begingroup$

Compute a distance metric between your two observed matrixes. Then to determine if that distance is significant, you use a permutation test. You can refer to this, it seems that your question is a duplicate of this one.

$\endgroup$
  • $\begingroup$ I have read all posts I could find regarding this question but the answers were still insufficient to me. Can you name an appropriate distance metric that I can use for the method you proposed? $\endgroup$ – bnd Feb 27 '17 at 17:38
  • $\begingroup$ The distance metric could be anything, you can take the L2 norm between the datasets and decide upon a threshold based on your problem. You can do this per feature based and add the distances up in the cost function. Take a look at normaldeviate.wordpress.com/2012/07/14/modern-two-sample-tests $\endgroup$ – saha rudra Feb 28 '17 at 2:45
0
$\begingroup$

Use the nearest- neighbour classifier on each point. If the two sets are from the same statistical source and have the same number of points, then you would expect all the data points to have a nearest neighbour with equal probability of belonging to each of the two groups. So a robust method, but computationally tedious, is to remove each point from the data set in turn and apply it to the to the remaining data using the nearest neighbour rule. This will find which of the two sets it most likely belongs to. Expect to get a near equal score. A simple coin-toss test with p = 0.5 and n, the total number of points will give you the likelihood of the sets being From the same source. This approach is valid for multi dimensional scatter plots with no assumption of normality, so it is very robust. No doubt it can be used where the two sets have different numbers where p would be represent the ratio.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.