Inconsistent normality tests: Kolmogorov-Smirnov vs Shapiro-Wilk I'm currently looking into some data that was produced by an MC simulation I wrote - I expect the values to be normally distributed. Naturally I plotted a histogram and it looks reasonable (I guess?): 
 
[Top left: histogram with dist.pdf(), top right: cumulative histogram with dist.cdf(), bottom: QQ-plot, data vs dist]
Then I decided to take a deeper look into this with some statistical tests. (Note that dist = stats.norm(loc=np.mean(data), scale=np.std(data)).) What I did and the output I got was the following:


*

*Kolmogorov-Smirnov test:  
scipy.stats.kstest(data, 'norm', args=(data_avg, data_sig))
KstestResult(statistic=0.050096921447209564, pvalue=0.20206939857573536)


*Shapiro-Wilk test:  
scipy.stats.shapiro(dat)
(0.9810476899147034, 1.3054057490080595e-05)
# where the first value is the test statistic and the second one is the p-value.


*QQ-plot:  
stats.probplot(dat, dist=dist)

My conclusions from this would be:


*

*by looking at the histogram and the cumulative histogram, I would definitely assume a normal distribution

*same holds after looking at the QQ plot (does it ever get much better?)

*the KS test says: 'yes this is a normal distribution'
My confusion is: the SW test says it is not normally distributed (p-value much smaller than significance alpha=0.05, and the initial hypothesis was a normal distribution). I don't understand this, does anyone have a better interpretation? Did I screw up at some point?  
 A: There are innumerable ways a distribution can differ from a normal distribution.  No test could capture all of them.  As a result, each test differs in how it checks to see if your distribution matches the normal.  For example, the KS test looks at the quantile where your empirical cumulative distribution function differs maximally from the normal's theoretical cumulative distribution function.  This is often somewhere in the middle of the distribution, which isn't where we typically care about mismatches.  The SW test focuses on the tails, which is where we typically do care if the distributions are similar.  As a result, the SW is usually preferred.  In addition, the KW test is not valid if you are using distribution parameters that were estimated from your sample (see: What is the difference between the Shapiro-Wilk test of normality and the Kolmogorov-Smirnov test of normality?).  You should use the SW here.  
But plots are generally recommended and tests are not (see: Is normality testing 'essentially useless'?).  You can see from all your plots that you have a heavy right tail and a light left tail relative to a true normal.  That is, you have a little bit of right skew.  
A: You can't cherry pick normality tests based on the results. In this case, you either go with the rejection in any test conducted, or not use them at all. KS test is not very powerful, it's not a "specialized" normality test. If anything SW is probably more trustworthy in this case. 
To me your QQ plot has signs of either fat right tail or skew to the left, or both. I would suggest using Tukey's tool to study the fatness of tails. It'll give you an indication how much a distribution is like normal or Cauchy.
