I'm interested in assessing if the marginalized posterior of a parameter obtained through a Bayesian MCMC process is "more or less normal".

I use quotation marks because I'm not trying to asses for normality strictly speaking. I've tried the usual tests and (as stated in Is normality testing 'essentially useless'? and Why would all the tests for normality reject the null hypothesis?) they invariably fail. I need a coarse parameter to tell me if the distribution looks "more or less normal". This is because I usually apply the MCMC to several datasets one after another, and I need a quick way to asses the "degree of non-normality" without having to check each one visually.

What I came up with is a very quick and simple test:

  1. Estimate mean, median, and mode for the distribution.
  2. Obtain the largest distance between any of these three values.
  3. Scale that distance by the range of the distribution to obtain the final $N_t$ parameter, bounded between [0, 1].

The idea is that as the distribution tends to normality then $mean\sim median\sim mode$ thus $N_t\rightarrow0$, and as the distribution becomes "less normal" then $N_t\rightarrow1$.

I've tested it and it seems to give reasonable results, but I'm concerned that I might be missing some fringe cases where this test will fail and get past me.

Is this a reasonable test for coarse normality? Should I instead rely on a more sophisticated test as the one described in What is a good index of the degree of violation of normality and what descriptive labels could be attached to that index??

  • $\begingroup$ The real question is: why use a test? First off, it's a departure from the Bayesian paradigm. If interest lies in the shape of the posterior, you can simply describe the mean, the median, the skewness, and any possible multimodality. Overlay a normal distribution as compared to a kernal smoothed density of the posterior as a descriptive figure. Testing made us forget how to describe things. If you have to have a test, use the Shapiro Wilk test. $\endgroup$ – AdamO Jan 17 '19 at 16:40
  • $\begingroup$ @AdamO I need a test to tell me quickly how "non-normal" (coarsely) a distribution is because I want to identify these distributions without having to check them visually. As I said in the question, all the usual tests for normality (including Shapiro-Wilk) give p=0 even if the distribution is very "normal-looking". This is apparently expected for large datasets. So neither of those tests is useful to me. $\endgroup$ – Gabriel Jan 17 '19 at 16:46
  • $\begingroup$ Nothing gives p=0 for continuous support test statistics, you have an issue with the precision of the test. Choose a stringent cut off or at least use the test statistic to rank distributions. Alternately, just choose a pragmatic rank based approach by taking the sum-squared residuals from the identity line in the QQplot. $\endgroup$ – AdamO Jan 17 '19 at 16:49
  • $\begingroup$ Your approach concentrates on the symmetry of the distribution, which is just one property of the normal distribution. There are many examples that that are symmetric but non-normal. $\endgroup$ – Balint T. Jan 17 '19 at 16:49

Here's a counterexample to your approach: mean = median = mode

enter image description here

If you sample from this distribution here is the QQ-plot:

enter image description here

And the Mean-squared displacement from the calibration line is:

n <- 1e6
p <- rbinom(n, 2, 0.5)
x <- numeric(n)
X <- cbind(rnorm(n, -2, 0.25), rnorm(n, 0, 0.25), rnorm(n, 2, 0.25))

for(i in 1:n) x[i] <- X[i, p[i]+1]

mean({sort(x) - qnorm(1:n/{n+1})}^2)

gives a stable constant that converges to a fixed, finite value as $n \rightarrow \infty$.

In this case: $\approx$ 0.32

  • $\begingroup$ Great example of my test failing Adam, thank you. Since I code in Python, I can use scipy.stats.probplot which already returns the coefficient of determination for the linear fit between my distribution and a normal probability plot. This should be similar to the "Mean-squared displacement" you used, I believe? $\endgroup$ – Gabriel Jan 17 '19 at 17:59
  • 1
    $\begingroup$ @Gabriel that's right. $\endgroup$ – AdamO Jan 17 '19 at 18:20
  • $\begingroup$ The square of this coefficient of determination is the Ryan-Joiner statistic, and is closely related to the Shapiro-Francia statistic $\endgroup$ – Glen_b Jan 18 '19 at 0:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.