# Can I use the Kolmogorov Smirnov Test in this manner to determine normality?

Hi I have the following Age data set for 69 people, I wish to determine if this is normal or not using the Kolmogorov Smirnov Test:

age <- c(67, 63, 45, 10, 8, 41, 75, 68, 63, 60, 55, 58, 58, 38, 40,
63, 70, 38, 40, 38, 36, 6, 42, 63, 65, 63, 65, 38, 40, 39, 38,
38, 40, 50, 42, 45, 16, 20, 41, 2, 38, 39, 36, 39, 34, 40, 44,
88, 39, 41, 35, 35, 68, 65, 36, 35, 38, 40, 42, 10, 8, 44, 38,
40, 68, 63, 19, 34, 38)


Can I actually synthetically generate normal data using synthetic_age <- rnorm(length(age), mean = mean(age), sd = sd(age)) and then subsequently compare my age data with the synthetically generated normal data as follows:

ks.test(age, synthetic_age)


I have plotted both sets on an ECDF and the age variable appears to be somewhat normal. I have also run the KS-test and it says that the two data sets (observed and synthetic) are from the same distribution, hence can I say that because one is normal, the other will be too?

I know about the Shapiro Wilk test, but I don't wish to use that here. Also it is telling me that the age data I have is not normal. I don't know which to believe.

• As a general principle, it is always worse to compare your data to synthetically generated data instead of comparing them to the underlying distribution, because using the synthetic data introduces unnecessary uncertainty in the result. The other problematic issues in this use of the KS test have been discussed in many other threads: please see stats.stackexchange.com/search?q=ks.test. – whuber May 29 '19 at 12:22
• Two questions: (a) Why do you want to know whether the age data are consistent with normal? (b) What is your objection to using the Shapiro-Wilk test? // Your specification of age will work better if observations are separated by commas: age = c(67, 63, 45, 10, 8, 41, 75, 68, 63, 60, 55, 58, 58, 38, 40, 63, 70, 38, 40, 38, 36, 6, 42, 63, 65, 63, 65, 38, 40, 39, 38, 38, 40, 50, 42, 45, 16, 20, 41, 2, 38, 39, 36, 39, 34, 40, 44, 88, 39, 41, 35, 35, 68, 65, 36, 35, 38, 40, 42, 10, 8, 44, 38, 40, 68, 63, 19, 34, 38). – BruceET May 29 '19 at 15:13
• You can. But normality tests tends towards rejecting the null hypotheis (normality) when working with real-world data pretty much every single time, as very few things match an actual normal distribution. When the hypothesis is not rejected, it's pretty much always because of a lack of data – David May 29 '19 at 20:34

Data descriptions seem inconsistent with normality: Formal tests aside, there are strong indications that your age observations are not a random sample from a normal population. In R, we obtain a boxplot and a normal probability plot (Q-Q plot):

age = c(67, 63, 45, 10,  8, 41, 75, 68, 63, 60,
55, 58, 58, 38, 40, 63, 70, 38, 40, 38,
36,  6, 42, 63, 65, 63, 65, 38, 40, 39,
38, 38, 40, 50, 42, 45, 16, 20, 41,  2,
38, 39, 36, 39, 34, 40, 44, 88, 39, 41,
35, 35, 68, 65, 36, 35, 38, 40, 42, 10,
8, 44, 38, 40, 68, 63, 19, 34, 38)

par(mfrow=c(1,2))
boxplot(age, col="skyblue2", pch=20)
qqnorm(age, pch=10)
abline(a=mean(age), b=sd(age))
par(mfrow=c(1,1))


Specifically, there are no observations between 21 and 33. This makes the lower quartile and the median almost equal (as shown in the boxplot) and accounts for the 'jump' in the normal probability plot.

summary(age);  sd(age)
Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
2.0    38.0    40.0    43.2    58.0    88.0
[1] 17.75038  # sd


It would be extremely rare for a sample of size $$n = 69$$ from a normal population with $$\mu$$ near 43.2 and standard deviation near 17.75 not to have any observations in such an interval. (Of course, each sample will have its own peculiarities, but this quirk seems too strange not to mention.)

 ( 1 -  diff( pnorm(c(21,33), mean(age), sd(age)) ) )^69
[1] 1.427818e-06


As you say, the Shapiro-Wilk test rejects the null hypothesis of normality at any reasonable level of significance:

shapiro.test(age)$p.value [1] 0.0007492721  The implementation of ks.test in R is inappropriate here because we do not know the parameters $$\mu$$ and $$\sigma$$ of the population distribution and because there are many ties among the 69 observations. (In addition to the links suggested by @whuber, please read the documentation of the R procedure ks.test carefully.) A histogram of the age data and the default kernel density estimate in R of the population make me wonder if your sample might have come from a mixture distribution of three components (i) people below 21 years of age, (ii) people about 40, and (iii) people about 65. hist(age, prob=T, br=30, col="skyblue2") lines(density(age), lwd=2, col="blue") curve(dnorm(x, mean(age), sd(age)), add=T, col="red", lwd=2, lty="dotted")  Confidence intervals for population mean. However, if you want to find a 95% confidence interval for $$\mu,$$ then a t confidence interval procedure would be sufficiently robust against non-normality to give a useful result. The sample size is moderately large, there is no evidence of extreme skewness in your sample and there are no extreme outliers. The 95% t confidence interval for $$\mu$$ (taken from the t.test procedure in R) is $$(38.84, 47,47).$$ By contrast, one type of 95% nonparametric bootstrap confidence interval for $$\mu$$ gives $$(39.12, 47.29)$$ and the nonparametric Wilcoxon 95% CI for the population median is $$(39.0, 49.0).$$ t.test(age)$conf.int
[1] 38.93879 47.46700
attr(,"conf.level")
[1] 0.95

set.seed(529)
d = replicate(10^5,  mean(sample(age, 71, rep=T)) - 43.2)
43.2 - quantile(d, c(.975,.025))
97.5%     2.5%
39.11831 47.28732

wilcox.test(age, conf.int=T, conf.lev=.95)\$conf.int
[1] 38.99994 49.00005
attr(,"conf.level")
[1] 0.95


Addendum: @whuber rightly points out the subjective nature of judging whether a particular normal Q-Q plot looks "too far" from normal. One method I have seen to provide a framework for such judgments is to compare the Q-Q plot of the sample in question with Q-Q plots of several samples of the same size sampled from a normal population with means and SDs that match the observed mean and SD of the sample in question.

In the figure below, we see that some points in the Q-Q plot of age (dark blue) seem to lie slightly beyond the cloud of points for Q-Q plots of 19 such comparison samples (light grey). [As with all simulations, different seeds may give slightly different results.]

set.seed(4321)
qqnorm(age, col="white") # to set up axes
for(i in 1:19) {
x = sort( rnorm(69, mean(age), sd(age)) )
points( qnorm(ppoints(1:69)), x, col="lightgrey", pch=20)
}
points(qnorm(ppoints(1:69)), sort(age),  col="blue", pch=19)

• (+1) Re "it would be extremely rare...:" please beware formal tests of post hoc hypotheses. The resulting small p-value is artificially small, most likely by orders of magnitude in this case. (That explains why the SW test p-value is almost a thousand times greater.) The hypothesis you would need to test is "what are the chances a Normal sample, when graphed as a probability plot, would lead me to believe the underlying distribution is not normal?" I have found it very instructive to spend some time examining simulated probability plots: they vary at random more than I had expected. – whuber May 29 '19 at 19:32
• @whuber. Yes, I was just now pondering how much fuss to make over this (21-33) gap, which is one of two that show in the histogram. I guess your cautionary comment and my added sentence, are sufficient to prevent anyone trying to turn this into a formal test. – BruceET May 29 '19 at 20:04
• Re the edit: very nice! Did you notice you're getting close to illustrating the KS test statistic (or even the SW test)? Just play the same simulation game with the ECDF (thereby bringing us full circle to the original question, which mentions a visual assessment of the ECDF). – whuber May 29 '19 at 22:00
• A thorough answer. I would just add, that the main argument that the data are not normal is IMHO simply: We know that age is non-negative.,All the measured values are integers. Normal distribution allows for negative and fractional values. Therefore not normal. Whether treating them as normal is a useful approximation then depends heavily on the context. – Martin Modrák May 30 '19 at 6:21