2
$\begingroup$

Disclaimer: Note this is a simplified version of my actual problem. These are not my real data.

The set up

The hexbin plot shows mass and height of goats in Iceland. There are a few thousand samples.

I measure the mass and height of goats in Japan. These are shown by the red dots. There are far fewer samples.

The problem

My hypothesis is the measurements of the Japanese goats and the Icelandic goats are from the same underlying sample.

I was going to use the Kolmogorov-Smirnov two-sample test to test for this, but one of the distributions is far from continuous. I can just use a chi-square goodness of fit test either.

Is there a different test I can do to show goat_Japan = goat_iceland?

enter image description here

Dave pointed me along the right lines in a comment to the question, and cleared up some confusion. I believe I found an answer (with a Python script to do it) here.

Improve this question
$\endgroup$
6
  • $\begingroup$ Do you want to compare the two-dimensional distributions? $\endgroup$
    – Dave
    Commented Sep 25, 2019 at 10:11
  • $\begingroup$ Yeah, and that is where I run into issues with the chi-square test, as I want to compare both 2D distributions. In Python, I want to write something like p_value = stat_test(distA['Height], distA['Mass'], distB['Height'], distB['Mass']) $\endgroup$
    – Sean Mooney
    Commented Sep 25, 2019 at 10:17
  • 1
    $\begingroup$ Another question: why do you say that one of the distributions is far from continuous? You have empirical distributions that are necessarily discrete. However, hypothesis testing cares about the populations from which the empirical distributions are drawn. Do you have reason to believe that one of the population distributions is discrete while the other is continuous? (I would say that you have no reason to believe that for your posted example. Your real data may be a different story.) $\endgroup$
    – Dave
    Commented Sep 25, 2019 at 10:30
  • $\begingroup$ Oh yeah, you are correct. I do believe the underlying distributions are continuous, my sampling of them is just discrete. $\endgroup$
    – Sean Mooney
    Commented Sep 25, 2019 at 10:33
  • 1
    $\begingroup$ The sampling is always discrete, so you can use tests for continuous populations. I wonder if either of these links answer your question: stats.stackexchange.com/questions/71036/… stats.stackexchange.com/questions/25946/… $\endgroup$
    – Dave
    Commented Sep 25, 2019 at 10:41

2 Answers 2

Reset to default
1
$\begingroup$

As mentioned by others, it looks like a classical case for two-dimensional Kolmogorov-Smirnov, first published by J. A. Peacock, Two-dimensional goodness-of-fit testing in astronomy, Monthly Notices Royal Astronomy Society 202 (1983) 615–627. Free PDF

Here few additional references not mentioned so far:

  1. Lopes et al. 'The two-dimensional Kolmogorov-Smirnov test'. Great discussion of all relevant implementation to date. Free PDF here.
  2. Recent fast implementation of the test by Xiao 'A fast algorithm for two-dimensional Kolmogorov–Smirnov two sample tests', Computational Statistics and Data Analysis 105 (2017) 53–58. link to PDF. R package
  3. Matlab implementation by Muir can be found here
  4. Cooke's algorithm mentioned by Lopes et al. can be found here
  5. and last but not least it's covered in Numerical Recipes link

Also wanted to mention the 'earth mover distance', EMD, https://en.wikipedia.org/wiki/Earth_mover%27s_distance which is an alternative solution. “EMD is a measure of the distance between two probability distributions over a region D. In mathematics, this is known as the Wasserstein metric.” Python code is available here.

Improve this answer
$\endgroup$
0
$\begingroup$

You say you want to "show goat_Japan = goat_iceland". If you are trying to prove that goat_Japan = goat_iceland, as your post implies, then you can't use the KS test. The KS test would use as null that goat_Japan = goat_iceland, and can only prove that this isn't true. It can't prove that it IS true: as with all statistical tests, you can only fail to reject the null, not prove the null.

So the fact that it is 2-dimensional, 2-sample, and observations are discrete, aren't an obstacle to using the KS test, as others have noted. But the KS test simply won't prove what you are setting out to prove.

Have a look here and here re the use of the KS test for Equivalence testing which is what it sounds like you want.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.