5
$\begingroup$

The q-q plot is a nice way to see a deviation from the normality. But this is only "eye estimation"

Can the R square of the linear regression be the effect size of the deviation from the normality? for any normality test, not necessarily the SW test...

Improve this question
$\endgroup$
4
  • $\begingroup$ Could be a good idea, see statlit.org/pdf/2012-jernigan-asa.pdf $\endgroup$
    – kjetil b halvorsen ♦
    Commented May 13, 2021 at 2:55
  • 1
    $\begingroup$ Thank you, Kjetil. This is a very nice article. I glad to see that more people thought of R as effect size to the normality. I searched and I didn't find any article about SW effect size calculation $\endgroup$
    – OB1
    Commented May 17, 2021 at 3:39
  • 1
    $\begingroup$ Hi Kjetil, when running simulations I noticed that the average R depends on the sample size. I would expect the same effect size when sampling the same distribution. Cohen's d dividing by the standard deviation, do you have any idea how to fix R to be the same for different sample sizes? Is this makes sense? For example: when checking Chi-squared(df=40): (repeates=10000) sample size = 1000, average(R)=0.99368 sample size = 100, average(R)=0.9886 sample size = 50, average(R)=0.98408 sample size = 20, average(R)=0.97371 sample size = 10, average(R)=0.96145 $\endgroup$
    – OB1
    Commented May 22, 2021 at 2:11
  • $\begingroup$ That is an interesting observation ... It could be that with a larger sample size, you get more values more far out in the distribution, where the non-linearity of the plot is more visible. So it ough't not to occur under the null ... ? $\endgroup$
    – kjetil b halvorsen ♦
    Commented May 22, 2021 at 3:04

1 Answer 1

Reset to default
5
$\begingroup$

The correlation coefficient in the qqplot can be used as a test for normality (in the case of a normal qqplot, or for some other null distribution model). See for instance this paper Developing a Test of Normality in the Classroom.

But if the correlation in the plot is useful as an effect size, is another matter ... test statistics by themselves are not necessarily useful as an effect size, it is better to start with an estimator of some parameter, but in this case of goodness of fit testing, which parameter?

The rest is a long comment to some of the OP comments above: Let us do some simulation in the case of the normal distribution, so tests of normality. Below is a plot of simulation with 1000 replications and various sample sizes:

Histograms of qqplot correlations from simulations

The OP noted that means increase with sample size. The plot shows the reason, there is a hard upper limit at 1, but a long lower tail, especially with lower sample size. So the histograms do not have the same form, they are skew for low $n$ and more symmetric for higher $n$. This in itself shows that the correlation cannot be a good measure of effect size.

For reference, below the R code used:

simcor <- function(N,  R,  simfun=rnorm, qfun=qnorm,  ...) { # N can be a vector
    res <- numeric(length=length(N))
    corrs <- matrix(as.numeric(NA), R, length(N))
    colnames(corrs) <- paste0("n=", N)
    for (i in seq_along(N)) {n <- N[i]
        vals <- matrix( simfun(n*R, ...), R, n)
        mean_corrs <- numeric(R)
        pp <- ppoints(n)  ;   qq <- qfun(pp, ...)
        for (r in 1:R) {
            samp <- vals[r, , drop=TRUE]
            mean_corrs[r] <- cor(sort(samp), qq)
        }
        res[i] <- mean(mean_corrs) ; names(res) <- N
        corrs[, i] <- mean_corrs
    }
    return(list(mean_corr=res,  corr=corrs))
}

run_norm <- simcor(c(10, 50, 100, 500, 1000),  1000)

xran <- c(0.77, 1)

 opar <-  par(mfrow=c(3, 2))
 for (i in seq_along(run_norm[[1]])) {
     hist(run_norm[[2]][, i], "FD", prob=TRUE, main=colnames(run_norm[[2]])[i],
          xlab=paste0("mean=", round(run_norm[[1]][i], 4)),
          xlim=xran) }
par(opar)
Improve this answer
$\endgroup$
5
  • $\begingroup$ Thanks again Kjetil :) Can you fix R to be an effect size? (say dividing by something) What can be the effect size for the deviation from the normality? $\endgroup$
    – OB1
    Commented May 23, 2021 at 11:07
  • $\begingroup$ I tried to divide by the sample standard deviation (R/S), but it didn't solve the problem ... $\endgroup$
    – OB1
    Commented May 23, 2021 at 11:50
  • $\begingroup$ The standard deviation of R is: S = sqrt( (1- R^2) / (n-2) ) $\endgroup$
    – OB1
    Commented May 24, 2021 at 1:19
  • $\begingroup$ I doubt that is the way to go! Why do you need an effect size? For what purpose? Anyhow, an effect size in this case must be some measure of deviation from normality, that could be many things ... skewness, kurtosis, or the test statistic from the KS test $\endgroup$
    – kjetil b halvorsen ♦
    Commented May 26, 2021 at 18:32
  • $\begingroup$ For every test you need both p-value and effect size, otherwise, you may get a significant result with a meaningless effect ... I believe that skewness and kurtosis are not as good as test effect size? KS statistic sounds good, but following problems: 1. D=0.9999999 for chi-squared(20) with n=20 (1 is normal distribution) 2. when running a simulation, the effect size is smaller for larger sample sizes, for example : chi(df=4): n=10=>0.84, n=20=>0.817, n=50=>0.794, n=100=> 0.78 The decline in effect size when increasing the sample size is not so strong, so maybe it is good enough ...? $\endgroup$
    – OB1
    Commented May 27, 2021 at 7:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.