Interpretation of QQ-plots

Question

I've read what they mean on a technical level, but I'm trying to build intuition related to a specific problem.

So I have some data, a set of measurements. It's pretty curvy, looking like a cross between a norma and a gamma maybe. I do a qqplot of it with a normal distribution. The qqplot looks reasonably linear, it has a high correlation coefficient. I do a qqplot with a gamma distribution; same thing. I do a qqplot with a binomial; same thing. At this point I guess suspicious. So I do a qqplot with a uniform distribution, which is blatantly wrong if you look at it. However, I still get back a correlation coefficient that's reasonably high (>0.95).

I'm wondering why this is, and if what I did sounds similar.

Fundamentally, what good is a qqplot if this is the case? And is there some other metric besides this correlation coefficient that would be a better indicator of whether the data fits a certain distribution. What I'm thinking of in particular is something like the sum of squared errors that we use in the caes of linear regressions. Is there any way to get the square errors of the data with the distribution we're trying to fit to it? Does this make sense?

Answer 1

Because the sorted data and the expected order statistics are always in the same order, the correlation will nearly always tend to be quite high even when the plot is not all that close to linear.

Nonetheless, the correlation can be a useful way to assess the closeness to/deviation from linearity in this instance -- you just have to adjust your idea of what a high correlation is. [Alternatively, you could look at $(1-R^2)$ as a measure of discrepancy; that would be easier to deal with.]

The squared correlation between the data and corresponding expected order statistics is essentially the Shapiro-Francia test statistic; if you use median order statistics instead, it's a test directly discussed by Filliben (1975) -- but asymptotically both of them (and the Shapiro-Wilk as well) will be the same (e.g. see Leslie, Stephens and Fotopoulos, 1986).

Filliben, J.J., (1975),
"The Probability plot correlation coefficient test for normality"
Technometrics, 17 (1) Feb. 111-117

Leslie, J.R., Stephens, M.A. and Fototpoulos, G. (1986),
"Asymptotic distribution of the Shapiro-Wilk W for testing normality"
Ann. Statist. 14 (3), 1497-1506

Shapiro, S.S. and Francia, R.S. (1972),
"An approximate analysis of variance test for normality"
JASA, 67 215-216,

Answer 2

If $n$ is large enough, then the normal distribution is a reasonable approximation of the binomial distribution. Thus not a big deal, that both fit the same data. With gamma distribution it of course depends on what gamma distribution we talk about, but you can approximate a normal distribution with a gamma distribution as well. With all three curves similar, it is not unlikely to get similar qqplots. Take a normal if x-values range from $-\infty$ to $+\infty$ and a beta if they are within a fixed range like $[0;1]$ or find some other criterion like this for the decision.