# Disagreement between normality tests and histogram graphs

My data consist of compaction measurements from 3 different cell types (X,Y, and Z). My goal is to know whether there are "significant" differences between these measurements, so I have tested for:

1. Whether my samples are normally distributed

• using the Shapiro–Wilk test
• using the Jarque-Bera test
• plotting qqnorm graphs
• plotting histograms
2. Whether the samples come from the same distribution

• using two-sample Kolmogorov–Smirnov test (K–S test) and comparing X vs Y, X vs Z, and Y vs Z
• using Kruskal–Wallis comparing X, Y, and Z together

My data consist of 232 measurements for X, 284 for Y, and 124 for Z. The Shapiro-Wilk and Jarque-Bera tests in R always give me p<0.05, which I accept as not being normally distributed. However, when I plot histograms I get a normal-like distribution. The qqnorm plots also don't look that skewed, but maybe this is just my inexperience in interpreting qqnorm graphs (this is my first time making them).   Because of the supposedly non-normal distribution, I compared my data using KS test and Kruskal-Wallis, which always give me the result that my population Z is drawn from a different distribution compared to X and Y. However, I do not know if this is true, as R always reports for my two-sample Kolmogorov–Smirnov tests:

Warning message:
In ks.test(dataX, dataY) : cannot compute correct p-values with ties

Warning message:
In ks.test(dataX, dataZ) : cannot compute correct p-values with ties

Warning message:
In ks.test(dataY, dataZ) : cannot compute correct p-values with ties


probably because the samples have different sizes.

I'd like to know what you think about it, and whether I should consider using more parametric tests rather than the non-parametric ones I've used, or whether the tests I've used are valid regardless of the normality of the data. Also, my measurements seem to differ very little among themselves, for example:

---Summary stats for WT cells
Min.    1st Qu. Median  Mean    3rd Qu. Max.
0.1450  0.3720  0.5000  0.5598  0.7102  1.9290

---Summary stats for Df cells
Min.    1st Qu. Median  Mean    3rd Qu. Max.
0.0550  0.4030  0.5445  0.5857  0.7210  1.5350

---Summary stats for Dp cells
Min.    1st Qu. Median  Mean    3rd Qu. Max.
0.0670  0.4790  0.6255  0.6782  0.7897  2.0160


Here's a boxplot of the data: Green=X, Blue=Y, Red=Z

So I'm unsure about the conclusions I may derive from them.

• Both the qq plots and the histograms look pretty skewed to me. Look at the right tails on the latter. Normally I think it's best to just trust your eyes, since 1) the issue of whether or not a distribution is normal often isn't terribly important (what usually matters is how far it deviates from normal), and 2) normality tests typically reject the null if you an appreciable amount of data. However, in this case I'd say your distributions are not normal.
– alex
Jan 24, 2013 at 22:47
• I haven't looked at the images: this is a general comment. Among all the tools commonly available to assess distributions, including histograms, kernel density estimates, QQ plots, boxplots, and even dot density plots, as well as the formal tests of distribution, histograms are perhaps the crudest and most arbitrary of them all (for reasons well documented in the literature). In the event of an apparent disagreement between a conclusion drawn from a histogram and a conclusion drawn through the correct application of any of these other methods, I would automatically favor the other method.
– whuber
Jan 25, 2013 at 0:25
• You may want to add a diagonal line to your qqplots with abline(b=1). If your data are in fact normal, the dots should lie on that line, so it will make it much easier to see any deviations from normality. Jan 25, 2013 at 9:25
• Thank you very much for these comments. I tried using the abline but it didn't work for me, supposedly I need to standardize my data for comparing against the 45 degree line. I used qqline instead (explained in my other comment on Charlie's answer) and it seems to be pretty in alignment with that qqline. Jan 25, 2013 at 17:06
• To aid in the reading of QQ-plots, I really like the Stabilized Normal Probability Plot (ref): smu.edu/-/media/Site/Dedman/Departments/Statistics/TechReports/…; jstor.org/stable/2335939?seq=1; or tandfonline.com/doi/abs/10.1080/… (The last one is an adaptation to BASIC that includes the information for plotting lines for various $\alpha$ values. Nov 7, 2020 at 21:35

It appears that your data can only take on positive values. In this case, the hypothesis of normality is often rejected. Normally distributed random variables range from positive to negative infinity, so only positive values would violate this. You could try taking the log of the observations and seeing whether these are normally distributed.

If your data follow a normal distribution, then the points in your QQ-plot should lie on a 45-degree line through the origin. Your plots do not look like that at all.

The KS test is giving an error because the distributions being tested are presumed to be continuous. In this case, the probability of witnessing two observations with the exact same value is 0. Your data set contains ties, invalidating this assumption. When there are ties, an asymptotic approximation is used (you can read about this in the help file). The error that you are receiving has nothing to do with data sets with different sizes.

In your post, you never specified the question that you are trying to answer--with sufficient precision, anyway. Do you really want to test that the distributions are the same? Would it be sufficient to test that the means are the same?

Unless you are willing to assume that the variables follow some distribution, there isn't much of an alternative to the KS test if you want to test for the distributions being the same. But there are several ways to test for differences in means.

• Thank you very much Charlie, you gave me great advice regarding my problem. I'll assume my data follows some distribution, so I guess the KS test with ties (thanks for this tip) is my only alternative. Because the mean is sensitive to outliers, I decided to test median values, which seem more robust, so would a Wilcoxon Signed-Ranks Test be useful? Finally (sorry for this much bothering!) I tested log values with qnorm and plotted qqline as well for each Jan 25, 2013 at 16:45
• qqnorm(logx);qqline(logx,col=2) And I get the following: i49.tinypic.com/2jexoiv.png i49.tinypic.com/2day3di.png i50.tinypic.com/sbrimh.png I read qqline passes through the 1st and 3rd quartiles of my sample, because otherwise for comparing against a 45 degree line wouldn't be useful as this is standardized, and for using it I'd have to standardize my own values. Is this correct? Or is it safe to assume my data (log val) is normally distributed? Also I don't know why using logs would be useful, how is it related to my original hypothesis testing? Thank you so much!! Jan 25, 2013 at 17:03
• Your links aren't working for me. Sometimes the log of a positive-only variable looks more normal-like (it undoes the skew in the data). Jan 25, 2013 at 18:39
• Does this mean the original (non-log) samples distribution is more normal-like? Here are the images. Thanks!! [link]img832.imageshack.us/img832/9397/screenshotat20130125115.png [link]imageshack.us/a/img37/9397/screenshotat20130125115.png [link]imageshack.us/a/img189/9397/screenshotat20130125115.png Jan 25, 2013 at 18:52
• No, what you provided was very not normal like. When you take the logs, many of the points lie on the 45-degree line. This suggests that the center of the distributions look normal-ish, but the patterns shown in the QQ-plots of the logs are consistent with a positive skew. Jan 25, 2013 at 22:50