# Quantifying QQ plot

The qq-plot can be used to visualize how similar two distributions are (e.g. visualizing the similarity of a distribution to a normal distribution, but also to compare two artibrary data distributions). Are there any statistics that generate a more objective, numerical measure that represent their similarity (preferably in a normalized (0 <= x <= 1) form)? The Gini coefficient is for example used in economics when working with Lorenz curves; is there something for QQ-plots?

As I say in response to your comment on your previous question, check out the Kolmogorov-Smirnov test. It uses the maximum absolute distance between two cumulative distribution functions (alternatively conceived as the maximum absolute distance of the curve in the QQ plot from the 45-degree line) as a statistic. The KS test can be found in R using the command ks.test() in the 'stats' library. Here's more information about its R usage.

• Note that (as I understand it), the K-S test is for testing empirical data against an a priori distribution. It's not appropriate for comparing two empirical distributions, nor is it appropriate to compare empirical data against an a priori distribution whose parameter values were estimated from the empirical data. Sep 21 '10 at 15:07
• @Mike, you can use the K-S test to compare two empirically derived distributions, see Charlie's prior answer and comments stats.stackexchange.com/questions/2918/lorenz-curve-qq-plot/… Sep 21 '10 at 15:39
• @Andy, Ah, I took point 3 from itl.nist.gov/div898/handbook/eda/section3/eda35g.htm as having the corollary that you can't compare two empirical CDFs, but I see that my assumption wasn't appropriate. Good to know, thanks! Sep 21 '10 at 16:02
• However, point 3 does imply that you can't use K-S to test whether your data come from a normal distribution with mean and sd estimated from the data. This is a popular error among the psychology students I meet. Sep 21 '10 at 21:28
• (+1) The superior aspect of this answer is that the K-S statistic can be read directly off the Q-Q plot.
– whuber
Apr 5 '17 at 23:10

I recently used the correlation between the empirical CDF and the fitted CDF to quantify goodness-of-fit, and I wonder if this approach might also be useful in the current case, which as I understand it involves comparing two empirical data sets. Interpolation might be necessary if there are different numbers of observations between the sets.

• Your paper includes very nice figures :)
– chl
Sep 21 '10 at 15:19
• @chi: They were all created in R using ggplot2. It's a fantastic graphics production system! Sep 21 '10 at 15:42
• What do you mean with fitted CDF? Sep 22 '10 at 15:44
• @Ampleforth, in that paper, I fit a distribution to empirical data, so by "fitted CDF" I meant the theoretical CDF of the fitted distribution. Sorry, I see how I could have been more clear! Sep 22 '10 at 17:06
• @user1243255 Ah, I see now that the link was to a paper in which I used this idea but didn’t show the actual EmpiralCDF-ExpectedCDF plots themselves. The new paper on SBC shows those though. And here’s a link to my paper: pubmed.ncbi.nlm.nih.gov/21139162 Apr 1 at 13:37

I would say that the more or less canonical way to compare two distributions would be a chi-squared test. The statistic is not normalized, though, and it depends on how you choose the bins. The last point can of course be seen as a feature, not a bug: choosing bins appropriately allows you to look more closely for similarity in the tails than in the middle of the distributions, for instance.

A pretty direct measure of the "closeness" to linearity in a Q-Q plot would be a Shapiro-Francia test statistic (which is closely related to the better known Shapiro-Wilk and can be regarded as a simple approximation to it).

The Shapiro-Francia statistic is the squared correlation between the ordered data values and the expected normal order statistics (sometimes labelled "theoretical quantiles") -- that is, it should be the square of the correlation you see in the plot, a pretty direct summary measure.

(The Shapiro-Wilk is similar but takes into account correlations between the order statistics; it has a similar interpretation to the Shapiro-Francia and is pretty much equally as useful as a summary of the Q-Q plot.)

Either way, for a single number summary of what the Q-Q plot shows, one of those could be a suitable way to summarize the plot.

Personally I tend to look more for deviation from linearity rather than nearness to it (which would suggest looking at $1-W'$). This scale tends to leave you with fairly constant values for a given amount of non-normality.

[Sometimes I multiply by $n$ ($1-W')$ tends to get smaller with $n$ if sampling a normal). Under sampling from a normal, the mean or median of $n(1-W')$ tend to be fairly stable as $n$ changes. Multiplication by $n$ is still not quite right though, it fractionally overcorrects -- the result increases with $n$ somewhere between $\log(n)$ and $\sqrt{\log(n)}$ -- but this variation is small compared to the sorts of values you tend to get with any kind of substantial deviation from normality. Getting to a scale where the distribution doesn't change much with $n$ makes it more like a transformed p-value (less useful as a measure of amount of non-normality, more useful if you're interested in something more like judging if it's not merely random variation).]