4
$\begingroup$

I have distributional data which I represent as a density. The data represents frequencies of user activities on a computer screen (e.g. amount of clicks on the y or x-axis of that screen but also other activities that can be related to coordinates and can therefore be binned by those coordinates (e.g. 5 pixels bins)). I would like to compare two kinds of that behavior and find out how compatible their distributions are. Very general. No assumptions exist. I can't assume parametric conditions such as linearity or normality.

I read about Lorenz curves and the Gini coefficient to be very much like what I need to compare distributions but also know that those methods find application primarily for economic and sociological problems and are usually not applied for general distributions. Am I applying the wrong tool for the job? What is your opinion about this? What alternatives do you recommend in order to find out how similar two distributions are?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

You can use a 2-sample Kolmogorov-Smirnov test to compare the two distributions. Other tests for comparing 2-samples are the Anderson-Darling test (although the 2-sample form of this is less frequently used), and the Baumgartner-Weiss-Schindler test.

Before you jump into hypothesis testing, though, you may want to graphically inspect the two distributions, either by overlaying their empirical CDF, or better by an empirical Q-Q plot.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Aren't these tests not restricted to hypotheses testing (testing the null hypothesis that they are the same)? I would like to have a similarity measure that quantifies their (dis)similarity in addition to simply testing IF they are (dis)similar and preferably in some comparable form (e.g. 20% dissimilar or 0.2 (i.e. a normalized value between 0 and 1 as a measure for similarity) etc.). Would you recommend to use the p value for that? Thanks in advance! $\endgroup$
    – Ampleforth
    Commented Sep 21, 2010 at 5:40
  • $\begingroup$ Ampleforth:> yes, you can use the p-value as a continuous (not just 0-1) and normalized measure of the differences between the two distributions. It'll be more accurate (in the sense of precise, or powerfull) than Gini's I. $\endgroup$
    – user603
    Commented Sep 21, 2010 at 8:29
  • 2
    $\begingroup$ +1 for the second suggestion. The Q-Q plot is one of the strongest and most insightful ways to compare two empirical distributions, because it graphically displays variations between the two throughout their range. It is also a natural graphical adjunct to the KS test, whose statistic is the maximum deviation between the Q-Q plot and its reference line (Y==X). $\endgroup$
    – whuber ♦
    Commented Sep 21, 2010 at 14:15
  • $\begingroup$ @whuber: good point about KS statistic being the maximum deviation. $\endgroup$
    – shabbychef
    Commented Sep 21, 2010 at 16:26
  • $\begingroup$ You can use the K-S test statistic as a measure of dissimilarity. For a two-sided test, the K-S test statistic is the largest absolute difference between the two distribution functions. As the distribution functions lie in [0,1], the difference also lies in [0,1]. If the two distributions differ slightly the p-value will tend to 0 as the sample sizes increase (since the power of the test increases) but the K-S test statistic will tend to a limit. $\endgroup$
    – onestop
    Commented Sep 21, 2010 at 16:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.