3
$\begingroup$

I have recently come across Q-Q plots and their usefulness in regards to visually inspecting whether a data sample follows a particular distribution.

Is there a way of quantifying the results of a Q-Q plot, to remove the subjectiveness of a visual inspection -- what looks very linear to some may look somewhat linear to others.

I have thought of two possible methods one could quantify this with.

  1. perform a linear fit on the Q-Q plot data and look at best fit statistics (e.g. chi-sqaured). Simulate data and look at the distribution of your fit statistics and see if the data sample's associated chi-square value is within a certain range of the simulated distribution of chi-squares.
  2. Again perform a linear fit and then determine confidence intervals e.g. $68\%$ and decide how many points are allowed to be outside this interval (again through simulation) to see if the sample should be rejected or not.

Is this appropriate? Of course I could use a distribution test, but I am loathed to go down the $p$-value avenue, and I especially want to avoid the $p < 0.05$ convention.

Improve this question
$\endgroup$
7
  • 2
    $\begingroup$ Various tests of normality, such as Shapiro-Wilk and Anderson-Darling essentially quantify linearity as in your first suggestion. But many statisticians with experience judging normality prefer to look at a Q-Q plot (preferably with a reference line). Especially for small sample sizes, it is important not to expect all points to lie near a line. For larger normal samples, you can expect a few points in the tails to stray from the line. $\endgroup$
    – BruceET
    Oct 23, 2020 at 15:45
  • 2
    $\begingroup$ You need to perform a correct linear fit. On a QQ plot it's essential to weight the residuals appropriately because (a) they are strongly heteroscedastic (they won't vary much in the middle but can vary appreciably in the tails) and (b) they are strongly autocorrelated. Apart from the KS test, all the good distributional tests (SW, AD, etc.) are essentially derived from good approximations to the covariance structure indicated by (a) and (b). Thus, corresponding to your favorite test there will be a linear fit to the QQ plot. $\endgroup$
    – whuber ♦
    Oct 23, 2020 at 16:05
  • 1
    $\begingroup$ No significance level is sacred. There's nothing wrong with using the 5% level, but then you'll reject 5% of truly normal samples, as in one of the four examples in my Answer. $\endgroup$
    – BruceET
    Oct 23, 2020 at 16:06
  • 2
    $\begingroup$ Perhaps: but on a positive note, it shows that there will exist a linear approximation to the plot that reflects the test you may be using and by plotting that line you can create a graphical representation of the test if you like. $\endgroup$
    – whuber ♦
    Oct 23, 2020 at 16:14
  • 1
    $\begingroup$ That's a good suggestion, thanks to you both for the input and advice. I think what I will do is use the $p<0.05$ not as a rejection criteria but more a "flag for visual inspection" via a Q-Q plot and what @whuber just suggested. $\endgroup$
    – user27119
    Oct 23, 2020 at 16:21

1 Answer 1

Reset to default
2
$\begingroup$

Here are Q-Q plots for four standard normal samples, two with $n=15$ and two with $n = 150.$

Shapiro-Wilk P-values are shown (left to right across rows); the test for data shown at lower left happens to be a false rejection. [If you use R, you can make additional examples by omitting my set.seed line.]

set.seed(1023)
par(mfrow=c(2,2))
x = rnorm(15); qqnorm(x); qqline(x, col="green")
 shapiro.test(x)$p.val
 [1] 0.480773
x = rnorm(15); qqnorm(x); qqline(x, col="green")
 shapiro.test(x)$p.val
 [1] 0.3259995
x = rnorm(150); qqnorm(x); qqline(x, col="green")
 shapiro.test(x)$p.val
 [1] 0.0392581   # Incorrectly fails at 4%
x = rnorm(150); qqnorm(x); qqline(x, col="green")
 shapiro.test(x)$p.val
 [1] 0.6917945
par(mfrow=c(1,1))

enter image description here

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.