The qq-plot can be used to visualize how similar two distributions are (e.g. visualizing the similarity of a distribution to a normal distribution, but also to compare two artibrary data distributions). Are there any statistics that generate a more objective, numerical measure that represent their similarity (preferably in a normalized (0 <= x <= 1) form)? The Gini coefficient is for example used in economics when working with Lorenz curves; is there something for QQ-plots?
4 Answers
As I say in response to your comment on your previous question, check out the Kolmogorov-Smirnov test. It uses the maximum absolute distance between two cumulative distribution functions (alternatively conceived as the maximum absolute distance of the curve in the QQ plot from the 45-degree line) as a statistic. The KS test can be found in R using the command ks.test()
in the 'stats' library. Here's more information about its R usage.
-
$\begingroup$ Note that (as I understand it), the K-S test is for testing empirical data against an a priori distribution. It's not appropriate for comparing two empirical distributions, nor is it appropriate to compare empirical data against an a priori distribution whose parameter values were estimated from the empirical data. $\endgroup$ Sep 21, 2010 at 15:07
-
4$\begingroup$ @Mike, you can use the K-S test to compare two empirically derived distributions, see Charlie's prior answer and comments stats.stackexchange.com/questions/2918/lorenz-curve-qq-plot/… $\endgroup$– Andy WSep 21, 2010 at 15:39
-
$\begingroup$ @Andy, Ah, I took point 3 from itl.nist.gov/div898/handbook/eda/section3/eda35g.htm as having the corollary that you can't compare two empirical CDFs, but I see that my assumption wasn't appropriate. Good to know, thanks! $\endgroup$ Sep 21, 2010 at 16:02
-
2$\begingroup$ However, point 3 does imply that you can't use K-S to test whether your data come from a normal distribution with mean and sd estimated from the data. This is a popular error among the psychology students I meet. $\endgroup$ Sep 21, 2010 at 21:28
-
1$\begingroup$ (+1) The superior aspect of this answer is that the K-S statistic can be read directly off the Q-Q plot. $\endgroup$– whuber ♦Apr 5, 2017 at 23:10
I would say that the more or less canonical way to compare two distributions would be a chi-squared test. The statistic is not normalized, though, and it depends on how you choose the bins. The last point can of course be seen as a feature, not a bug: choosing bins appropriately allows you to look more closely for similarity in the tails than in the middle of the distributions, for instance.
I recently used the correlation between the empirical CDF and the fitted CDF to quantify goodness-of-fit, and I wonder if this approach might also be useful in the current case, which as I understand it involves comparing two empirical data sets. Interpolation might be necessary if there are different numbers of observations between the sets.
-
-
$\begingroup$ @chi: They were all created in R using ggplot2. It's a fantastic graphics production system! $\endgroup$ Sep 21, 2010 at 15:42
-
-
$\begingroup$ @Ampleforth, in that paper, I fit a distribution to empirical data, so by "fitted CDF" I meant the theoretical CDF of the fitted distribution. Sorry, I see how I could have been more clear! $\endgroup$ Sep 22, 2010 at 17:06
-
1$\begingroup$ @user1243255 Ah, I see now that the link was to a paper in which I used this idea but didn’t show the actual EmpiralCDF-ExpectedCDF plots themselves. The new paper on SBC shows those though. And here’s a link to my paper: pubmed.ncbi.nlm.nih.gov/21139162 $\endgroup$ Apr 1, 2021 at 13:37
A pretty direct measure of the "closeness" to linearity in a Q-Q plot would be a Shapiro-Francia test statistic (which is closely related to the better known Shapiro-Wilk and can be regarded as a simple approximation to it).
The Shapiro-Francia statistic is the squared correlation between the ordered data values and the expected normal order statistics (sometimes labelled "theoretical quantiles") -- that is, it should be the square of the correlation you see in the plot, a pretty direct summary measure.
(The Shapiro-Wilk is similar but takes into account correlations between the order statistics; it has a similar interpretation to the Shapiro-Francia and is pretty much equally as useful as a summary of the Q-Q plot.)
Either way, for a single number summary of what the Q-Q plot shows, one of those could be a suitable way to summarize the plot.
Personally I tend to look more for deviation from linearity rather than nearness to it (which would suggest looking at $1-W'$). This scale tends to leave you with fairly constant values for a given amount of non-normality.
[Sometimes I multiply by $n$ ($1-W')$ tends to get smaller with $n$ if sampling a normal). Under sampling from a normal, the mean or median of $n(1-W')$ tend to be fairly stable as $n$ changes. Multiplication by $n$ is still not quite right though, it fractionally overcorrects -- the result increases with $n$ somewhere between $\log(n)$ and $\sqrt{\log(n)}$ -- but this variation is small compared to the sorts of values you tend to get with any kind of substantial deviation from normality. Getting to a scale where the distribution doesn't change much with $n$ makes it more like a transformed p-value (less useful as a measure of amount of non-normality, more useful if you're interested in something more like judging if it's not merely random variation).]
-
$\begingroup$ This answer kept me intrigued these days. However, I still couldn't understand the motivation behind the Shapiro-Francia test. I failed to see how it's measuring the deviation of being a straight line in the QQ plot. I'd appreciate if you could elaborate on that.. $\endgroup$– StudentSep 28, 2023 at 17:09
-
1$\begingroup$ It's essentially a simple function of the correlation in the plot. The Ryan-Joiner-Ryan test is another closely related test, even more directly related to the correlation. $\endgroup$– Glen_bSep 29, 2023 at 2:51