# Need reasons or references on small p-values with large data sets

What are reasons or references for a statement such as "When you have a lot of data the statistical problem you run into is that even a tiny difference will be statistically significant." I see this quite often and I need references as to why this is true. Any help is appreciated.

The reason is that as sample size increases your ability to detect deviations from the null (no effect) increases [standard errors of the estimate shrink]. Therefore, at sufficiently large sample sizes, even tiny effects may become statistically significant (i.e. $p < \alpha$). It is always (or at least almost nearly always) an appropriate thing to do to look at the effect size of your statistic. The effect size will quantify how large your effect is, hopefully in terms you care about (e.g. proportion of variance accounted for or magnitude of average difference scaled by error in measurement). If you know how to interpret your effect size, then you will also know how to judge whether this is a tiny effect of no theoretical interest, or an effect large enough to be worthy of consideration.