I have recently come across Q-Q plots and their usefulness in regards to visually inspecting whether a data sample follows a particular distribution.
Is there a way of quantifying the results of a Q-Q plot, to remove the subjectiveness of a visual inspection -- what looks very linear to some may look somewhat linear to others.
I have thought of two possible methods one could quantify this with.
- perform a linear fit on the Q-Q plot data and look at best fit statistics (e.g. chi-sqaured). Simulate data and look at the distribution of your fit statistics and see if the data sample's associated chi-square value is within a certain range of the simulated distribution of chi-squares.
- Again perform a linear fit and then determine confidence intervals e.g. $68\%$ and decide how many points are allowed to be outside this interval (again through simulation) to see if the sample should be rejected or not.
Is this appropriate? Of course I could use a distribution test, but I am loathed to go down the $p$-value avenue, and I especially want to avoid the $p < 0.05$ convention.