# Statistically significant interpretation

I'm looking for confirmation on my understanding of a 'statistically significant' effect.

For example, let's say that eating apples has a statistically significant positive effect on health (given that you can quantify health in some way). This does not mean that everyone will benefit from eating apples. Rather, it means the average individual from the population will benefit from eating apples.

This would imply that that the closer a single individual is to the the mean, the more likely they are to benefit from eating apples. The farther away an individual is from the mean, the less probable the tangible benefit from eating apples becomes.

Finally, if the above is true, how can statistically significant effects be applied at the individual level when individual effects require comparison to the mean? For example, the statistically significant result for apples is only relevant insofar that you are near the mean, but we have no idea how far any individual is from the mean without some additional metric. Is there a metric for this?

I suspect this line of thought is related to Stephen Jay Gould's "The Median Isn't the Message".

• The conclusion supported by the significant effect is that the mean effect is not 0. This doesn't imply what portion of the distribution will be benefited. It could be it benefits only those in the lowest 1% of the distribution. It doesn't imply those close to the mean will be affected. Jan 11, 2017 at 18:07
• The lowest 1% of the population distribution, not the distribution of sample means right? If so, how are p-values useful if they ignore the population distribution? I understand how the CLT guarantees a normal distribution of the sample means but that doesn't seem very useful if we ultimately care about the population distribution. Jan 11, 2017 at 19:31
• Also if the population distribution is normal, is it possible to have a significant effect if only the bottom 1% of the distribution is affected? Jan 11, 2017 at 19:37
• @David Not all tests that assume normality are robust to violations of it. You can certainly make an argument that t-tests are somewhat level-robust, but that's hardly the gamut of normal-theory tests (consider F-tests for equality of variance for example), and level robustness alone may not be sufficient -- it depends on the circumstances: the power properties (e.g. as indicated by relative efficiency against common alternative choices) may become poor in large samples when looking for small effects, which may or may not be a big issue. Jan 12, 2017 at 1:18
• @glen_b The quantile of what exactly? What distribution are we talking about? For example, the top 10 percent of those affected by apple eating? Or the top 10 percent of some other variable in the population (like health, wealth, etc.)? If the former, we may only see the effect in the top 10 percent of the treatment effect? Jan 20, 2017 at 3:11