14
$\begingroup$

The qq-plot can be used to visualize how similar two distributions are (e.g. visualizing the similarity of a distribution to a normal distribution, but also to compare two artibrary data distributions). Are there any statistics that generate a more objective, numerical measure that represent their similarity (preferably in a normalized (0 <= x <= 1) form)? The Gini coefficient is for example used in economics when working with Lorenz curves; is there something for QQ-plots?

$\endgroup$

4 Answers 4

9
$\begingroup$

As I say in response to your comment on your previous question, check out the Kolmogorov-Smirnov test. It uses the maximum absolute distance between two cumulative distribution functions (alternatively conceived as the maximum absolute distance of the curve in the QQ plot from the 45-degree line) as a statistic. The KS test can be found in R using the command ks.test() in the 'stats' library. Here's more information about its R usage.

$\endgroup$
6
  • $\begingroup$ Note that (as I understand it), the K-S test is for testing empirical data against an a priori distribution. It's not appropriate for comparing two empirical distributions, nor is it appropriate to compare empirical data against an a priori distribution whose parameter values were estimated from the empirical data. $\endgroup$ Sep 21, 2010 at 15:07
  • 4
    $\begingroup$ @Mike, you can use the K-S test to compare two empirically derived distributions, see Charlie's prior answer and comments stats.stackexchange.com/questions/2918/lorenz-curve-qq-plot/… $\endgroup$
    – Andy W
    Sep 21, 2010 at 15:39
  • $\begingroup$ @Andy, Ah, I took point 3 from itl.nist.gov/div898/handbook/eda/section3/eda35g.htm as having the corollary that you can't compare two empirical CDFs, but I see that my assumption wasn't appropriate. Good to know, thanks! $\endgroup$ Sep 21, 2010 at 16:02
  • 2
    $\begingroup$ However, point 3 does imply that you can't use K-S to test whether your data come from a normal distribution with mean and sd estimated from the data. This is a popular error among the psychology students I meet. $\endgroup$ Sep 21, 2010 at 21:28
  • 1
    $\begingroup$ (+1) The superior aspect of this answer is that the K-S statistic can be read directly off the Q-Q plot. $\endgroup$
    – whuber
    Apr 5, 2017 at 23:10
5
$\begingroup$

I would say that the more or less canonical way to compare two distributions would be a chi-squared test. The statistic is not normalized, though, and it depends on how you choose the bins. The last point can of course be seen as a feature, not a bug: choosing bins appropriately allows you to look more closely for similarity in the tails than in the middle of the distributions, for instance.

$\endgroup$
5
$\begingroup$

I recently used the correlation between the empirical CDF and the fitted CDF to quantify goodness-of-fit, and I wonder if this approach might also be useful in the current case, which as I understand it involves comparing two empirical data sets. Interpolation might be necessary if there are different numbers of observations between the sets.

$\endgroup$
9
  • $\begingroup$ Your paper includes very nice figures :) $\endgroup$
    – chl
    Sep 21, 2010 at 15:19
  • $\begingroup$ @chi: They were all created in R using ggplot2. It's a fantastic graphics production system! $\endgroup$ Sep 21, 2010 at 15:42
  • $\begingroup$ What do you mean with fitted CDF? $\endgroup$
    – Ampleforth
    Sep 22, 2010 at 15:44
  • $\begingroup$ @Ampleforth, in that paper, I fit a distribution to empirical data, so by "fitted CDF" I meant the theoretical CDF of the fitted distribution. Sorry, I see how I could have been more clear! $\endgroup$ Sep 22, 2010 at 17:06
  • 1
    $\begingroup$ @user1243255 Ah, I see now that the link was to a paper in which I used this idea but didn’t show the actual EmpiralCDF-ExpectedCDF plots themselves. The new paper on SBC shows those though. And here’s a link to my paper: pubmed.ncbi.nlm.nih.gov/21139162 $\endgroup$ Apr 1, 2021 at 13:37
2
$\begingroup$

A pretty direct measure of the "closeness" to linearity in a Q-Q plot would be a Shapiro-Francia test statistic (which is closely related to the better known Shapiro-Wilk and can be regarded as a simple approximation to it).

The Shapiro-Francia statistic is the squared correlation between the ordered data values and the expected normal order statistics (sometimes labelled "theoretical quantiles") -- that is, it should be the square of the correlation you see in the plot, a pretty direct summary measure.

(The Shapiro-Wilk is similar but takes into account correlations between the order statistics; it has a similar interpretation to the Shapiro-Francia and is pretty much equally as useful as a summary of the Q-Q plot.)

Either way, for a single number summary of what the Q-Q plot shows, one of those could be a suitable way to summarize the plot.

Personally I tend to look more for deviation from linearity rather than nearness to it (which would suggest looking at $1-W'$). This scale tends to leave you with fairly constant values for a given amount of non-normality.

[Sometimes I multiply by $n$ ($1-W')$ tends to get smaller with $n$ if sampling a normal). Under sampling from a normal, the mean or median of $n(1-W')$ tend to be fairly stable as $n$ changes. Multiplication by $n$ is still not quite right though, it fractionally overcorrects -- the result increases with $n$ somewhere between $\log(n)$ and $\sqrt{\log(n)}$ -- but this variation is small compared to the sorts of values you tend to get with any kind of substantial deviation from normality. Getting to a scale where the distribution doesn't change much with $n$ makes it more like a transformed p-value (less useful as a measure of amount of non-normality, more useful if you're interested in something more like judging if it's not merely random variation).]

$\endgroup$
2
  • $\begingroup$ This answer kept me intrigued these days. However, I still couldn't understand the motivation behind the Shapiro-Francia test. I failed to see how it's measuring the deviation of being a straight line in the QQ plot. I'd appreciate if you could elaborate on that.. $\endgroup$
    – Student
    Sep 28, 2023 at 17:09
  • 1
    $\begingroup$ It's essentially a simple function of the correlation in the plot. The Ryan-Joiner-Ryan test is another closely related test, even more directly related to the correlation. $\endgroup$
    – Glen_b
    Sep 29, 2023 at 2:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.