Difference between QQ plot and KS test

Question

While solving some exercises for HW, I came up to use QQ plot and KS test for the sake of testing goodness of fitting, where I also came up with the following doubts:

Precisely, let $X, Y \sim \text{Uniform}[0,1]$. Now, let's apply Box muller transformation to $X,Y$ and generate Normal distribution from it.

Now, by taking $X$ or $Y$, plot theoretical density and empirical density of uniform distribution [0,1].

Then again, define as $Z$ the box-muller transformation. And now, draw QQ plot from it.

In KS test, we see the sup of y axis distance between empirical cdf and theoretical cdf.

While in QQ plot, we check how far points are located with respect to the diagonal line (which is distance also). And let's consider this diagonal line as shifted and weirdly scaled cdf of uniform distribution, since cdf of uniform distribution is also diagonal.

Then, these two tests are indeed different? The underlying similarity makes me feel like that both tests are essentially the same.

I might be wrong...

Answer 1

Let's look at the two plots and then see how to convert one to the other. This is a sample of 30 standard normal random values:

The red curve in the left plot is a standard normal cdf - the theoretical model being used in this example - and the red curve on the right is the y=x line rather than the usual fitted line based on the data.

Now a Kolmogorov-Smirnov test finds the greatest vertical distance between the sample cdf (ecdf) and the theoretical for a completely specified distribution (a Lilliefors test would find the same distance but for a fitted distribution; it has smaller critical values as a result).

The position of that greatest vertical distance is marked. You could convert the red curve to a straight line by applying a standard normal cdf to the data. If the model is correct should convert it to a standard uniform), producing something rather close to an empircal-vs-theoretical P-P plot.

On the other hand, if we stretch that vertical axis in the ECDF by applying an inverse normal cdf, it will also straighten out the line. Unfortunately, the rightmost cdf value (that of the last data value) will end up stretched off to infinity. We don't expect the largest observation to be at infinity. So instead of tranforming the ecdf itself, we modify it slightly yield something more like the expected order statistic. Typically this is achieved by replacing $i/n$ for the $i$th value by $\frac{i-\alpha}{n+1-2\alpha}$ for some $\alpha$ in $[0,1)$. Many programs use $\alpha=\frac{3}{8}$, though R (which I used for the plots above) only uses that for small $n$; for $n>10$ it uses $\alpha=\frac12$. This treats the largest and smallest values symmetrically, stretching them the same distance from the center (and similarly for each pair further in).

With that change to the ecdf (from $u_i=i/n$ to $u_i^*=\frac{i-\alpha}{n+1-2\alpha}$) and then the transformation by the inverse of the normal cdf, we then have a Q-Q plot (though with the axes transposed -- that is with the theoretical quantiles on the y-axis rather than the x-axis).

Because of that stretching the distances measured by the K-S test are not preserved.

Is we instead stick to a P-P plot (but without the conversion to $u_i^*$, and again keeping the transformed observed data on the x-axis), where we squish up the x-axis instead of stretching the y-axis, that doesn't change the vertical distances, and that will preserve the Kolmogorov-Smirnov distance.

A more typical measure of fit used with a Q-Q plot would be based on the correlation. Indeed the Shapiro-Francia statistic (closely related to the Shapiro Wilk) and the quite similar Ryan-Joiner-Ryan statistic both look at something related to the correlation in the right hand plot.