# Sampling distribution need not be bell shaped?

Considering the following sample $X =$ [32.5, 31.7, 29.2, 28.8, 29. , 28.9, 29.9, 30.4, 26.9, 26.5]

I planned to evaluate the hypothesis of this sample having been generated from a normal distribution with an hypothetic $\mu$ and $sigma^2$ trough the use of the Kolmogorov Smirnov test statistic.

My strategy was to solve this problem using bootstrapping, so for each simulation I sampled from $X$ and computed the functional $T(X^*)$ where $T$ is the maximum distance from the empirical distribution of $X^*$ and the theoretical distribution of the distribution I considered. After all the simulations were through I had enough data to plot the empirical distribution of the t statistic, which I am right to call sampling distribution right?!

But when plotting an histogram of this same sampling distribution of the kolmogorov smirnov test statistic I got the following graphic:

My sample size is very low and some discretization was made to the observation, fixed precision. Is this distribution ok and perfectly normal? I am trying to understand if this can happen of (I speculate that it can), or if I messed up during the implementation of the ks-test.

Here's my implementation of the ks-test in R:

$H_0: Sample ~ N(\mu, \sigma^2)$

kstest = function(Xsample, mu, var) {

n = length(Xsample)
X_sample = sort(Xsample)

#List of differences
diffs = numeric(2*n)

for(j in 1:n){
f = pnorm(X_sample[j],mean=mu,sd=sqrt(var))
diffs[j*2-1] = j/n - f
diffs[j*2] = f - (j - 1)/n
}
return(max(diffs))
}

• You might have made a coding mistake , possibly there is some statistical error in the design of your code, or maybe your description of what you have done is ambiguous. When I run a quick simulation with small samples ($n=3$ and $n=8$), I get the expected uniform distribution of p-values and a continuous-looking distribution of KS statistics. Unless you provide details of your procedure, we cannot do any more than speculate. – whuber Nov 1 '16 at 13:47
• Thank you for your feedback, I updated my question. I am seriously considering that the problem is lack of precision in my samples.. or else there is something I am missing with the process. – Ramalho Nov 1 '16 at 15:09
• There appear to be some issues with your implementation of kstest. Compare it to the output of stats::ks.test. (This will not affect the shape of your histogram much, though.) – whuber Nov 1 '16 at 16:34
• Thank you!! I got it corrected now but the behaviour/shape is the same as you have suggested. Do you have any idea of what is causing it? Might it be the small sample? fixed precision? Can the sampling distribution assume different shapes other than bell curves? We know that, that is the expected behaviour when the statistic is the mean (CLT), but does the same thing need to apply to other statistics? Thank you a lot! – Ramalho Nov 1 '16 at 20:28
• Even the bootstrap distribution of the mean can exhibit severe multimodality, especially for small datasets. You might find it illuminating to make some normal QQ plots for a few bootstrap samples. – whuber Nov 1 '16 at 21:26

As @whuber suggested the bootstrap distribution, sampling distribution, can exhibit strong multimodality and points of discontinuity. For the sample mean there are only $C^{2N-1}_{N-1}$ possible values that $\bar{X}^*$ can take, this means that such a small sample as mine will have a very discrete behaviour like above. Fortunately as $N \rightarrow \infty$ the sampling distribution becomes more continuous.