What's the recommended approach when using hypothesis tests on a large sample dataset (1M+)? What's the recommended approach when using hypothesis tests on a large sample dataset? Talking about 1M+ records. 
According to this post (http://blog.minitab.com/blog/statistics-and-quality-data-analysis/large-samples-too-much-of-a-good-thing), having 1M+ records can make almost anything statistically significant. Are you supposed to sample a small fraction of the data for the hypothesis test?
 A: One way start is to ask the right questions. So e.g. instead of checking if the true means differ, you could ask if the true means differ more than some relevant threshold. Or instead of investigating if there is a true non-zero correlation, you could test if the true correlation is larger than some trivial bound.
The easiest way to test hypotheses in this way is to calculate confidence intervals on the parameter of interest and see if it excludes the "trivial" range of values.
Sometimes, even with huge data, classic approaches are perfectly fine. Think e.g. about deciding if a random number generator's output is non-uniform etc.
In many cases, the data at hand cannot be considered as random sample (e.g. because it is a convenience sample or the full population). In such settings, there is no real point in using classic stats tools and the often, the best way is to present a purely descriptive analysis instead of showing wrong/biased inductive stats.
A: Regardless of sample size, hypothesis testing should always be followed up by examining the domain relevance of the observed changes. You correctly point out that even arbitrarily small differences can become statistically significant with a large enough sample. But that doesn't indicate that the difference will have any meaningful impact on the system you're trying to model.
Finding a correlation significantly different from 0 may not really matter if the magnitude of the correlation is 0.0001. Finding a gene that's significantly upregulated with a fold change of 1.0002 probably won't have much impact on biology. A new drug treatment may yield a statistically significant survival benefit with a clinically insignificant magnitude of 1 hour. P-values alone don't tell the whole story, you need to assess the magnitude of the quantity being estimated.
