If a sample is not normally distributed, can a subset of the sample be normal?

Question

I have used a Shapiro-wilk test on all of my data and the results show that it is not normally distributed. However, could this mean that a subset of my data could still be normally distributed?

Answer 1

Sure it can: To see this, all you need to do is ask the equivalent question: if I started with a set of values that are normally distributed, could I add more values that stuff this up? Obviously the answer to this question is yes, and since the former set is the subset of the whole, your answer follows.

Answer 2

Yes, and here's an example. Let $X = Z X_1 + (1 - Z) X_2$, where $Z\sim Bern(0.5)\in\{0,1\}$, $X_1\sim N(5,1)$, and $X_2\sim N(-5,1)$, all independently of one another. Then $X$ is non-normal but if you condition on $Z=0$ or $Z=1$, which is like taking a subset of your full data, then $X$ is conditionally normal. But, as was mentioned in the comments, don't conflate a statistical test for normality with the true, underlying distribution. It's trivially true but generally uninteresting and potentially dangerous to observe that you can reject a null hypothesis of normality on your full data while, at the same time, fail to reject a null hypothesis of normality on a smaller subset of that data.

Answer 3

"Normality" is a property of an infinite population (potentially), not of a sample. Rather, a sample may be more or less consistent with the population having been normal, or it may be quite inconsistent with that.

I have used a Shapiro-wilk test on all of my data and the results show that it is not normally distributed.

The rejection would indicate that the data are not consistent with being a random sample from a population that has a normal distribution.

This is not saying much, since if you have enough data you're virtually certain to reject a null, and in most cases you can know for sure that the population you're sampling cannot possibly be actually normal.

However it's important to keep in mind that nearly always, populations will not actually be normal.

However, could this mean that a subset of my data could still be normally distributed?

If your data have subsets that come from different populations, then the combined data will behave like it's drawn from a mixture distribution; if those subpopulations differ, the data won't tend to look like they were drawn from a single normal population even if every subpopulation was normal.

[If instead you mean "can I choose elements - not a pre-existing group - from my data such that this subset would not be rejected by a normality test" the answer will often be yes, but such a thing would serve no purpose I can think of.]

Answer 4

Sure it can. As an example, here is R code that will generate a distribution where half the values are normal while others are not:

library(ggplot2)

# Create two pure distributions
n=1000
normals = data.frame(measurement=c(rnorm(n)), source=c(rep('normal', n)))
uniform = data.frame(measurement=c(runif(n)), source=c(rep('uniform', n)))

# Mix them
combined = rbind(normals, uniform)
combined$source = 'mixed'

# Make dummy dataframe for plotting
d = rbind(combined, normals, uniform)

# Plot data
p = ggplot(d) +
  geom_histogram(aes(x=measurement, fill=source)) +
  facet_grid(source~., scales = 'free_y') +
  theme_classic() + 
  theme(legend.position = 'none')

show(p)

# Do statistical tests
print(shapiro.test(normals$measurement))
print(shapiro.test(uniform$measurement))
print(shapiro.test(combined$measurement))

I won't reproduce the full output here for brevity, but the plot looks like this:

And the results of the Shapiro-Wilk test will not come as a surprise:

• $p=0.73$ for the normals
• $p=8.10^{-16}$ for the others
• $p=2.10^{-16}$ for the mix

This must be analogous to the situation you are envisioning.

Now from looking at the pink, it is easy to suspect that this a normal plus some junk, as evidenced by the "hump" in the middle. In fact, in this case the symmetry of the normal makes it seem easy to "clean up" most of the non-normals. So why not just do that?

The first problem is that as you can see from the green, even the pure normal isn't perfectly symmetric. That is because actual samples from the normal only converge to the idealized bell-curve shape as the number of samples approaches infinity. So you can't actually say how much of the "hump" is coming from the blue, and how much is just artifacts from the green (ie. "unlucky" samples of the normal). So you can't clean up the data precisely, you can only filter it so as to make it show what you want to show, in which case your analysis would be describing not some phenomenon in the real world, but something from your fantasy. If you sample something, but keep only values you like - what is the difference between that, and just fabricating all the values from your imagination? And how useful is the latter?

The second problem is that probably your goal is not to show that in your distribution, it is possible to cherrypick some subset of numbers that are distributed normally. Probably you are doing the SW in order to apply other methods, such as T-test, which require normality. They require this because all tests come with assumptions about the data. Without the assumptions, the test is impossible to mathematically derive. The test logically follows only at times when the assumptions are true; when the assumptions are false the test can say nothing about the situation. So the more your situation deviates from those assumptions, the less applicable the test becomes. The test will always faithfully reproduce some p-value no matter what data you plug in, but if you've completely violated the assumption, that p-value will lose all connection to reality. You will predict things with certainty based on that p-value, and those predictions will just never seem to come true.

Why are assumptions a problem in this context? Typically, one of them is that your data was randomly sampled. For example, if you are measuring the heights of people, it is assumed you didn't preferentially measure taller people because you were funded by the Tall People Association. If you throw out parts of your data that don't fit your expectation of normality, you are clearly not sampling randomly. The commonly used tests then do not apply, you must find tests that assume a non-random sample, which will be vastly more complicated and less useful.

So to recap, yes, it is possible that a subset of a sample can be normal. In fact, any real-valued distribution can be sampled and subsetted to leave a "normally distributed" set of numbers. But unfortunately, this concept cannot be easily exploited to "clean up" non-normal distributions and make them compatible with statistical methods that require normality.

Answer 5

Sure. In fact, you can easily have two subsets that are exhaustive and both normal. E.g.

set.seed(1234)  #Sets a seed


x1 <- rnorm(1000, 10, 10)  #Normal, N = 1000, mean 10, sd 10
x2 <- rnorm(1000, 25, 2)  #Normal, N = 1000, mean 25, sd 2

x <- c(x1, x2)

plot(density(x1), ylim = c(0, .25))  #Normal
lines(density(x2), col = "red") #Normal
lines(density(x), col = "green") #Not normal

Answer 6

Adding to other answers, or maybe saying it in different words, when we say that the "sample is normally distributed", we mean that we assume the we are talking of independent and identically distributed random variables. If we draw a subset of this sample using any sampling method that does not depend on the values, then the distribution of the subset will also be normal.

As about normality tests, they do not "prove" normality and are quite controversial, as you can learn from the Is normality testing 'essentially useless'? thread.