# Why are correlations acceptable in the small but considered "data dredging" in the large? [duplicate]

This question is going to be a bit more vague than I usually ask on StackExchange sites, but it keeps coming up in my life so I'm going to ask it.

Suppose I have a dataset of N datapoints that are each a vector of K values. Then for each pair of columns of the data I compute some association measure like correlation, and for every pair whose association measure exceeds some threshold I declare, "These fields are associated---something's going on here!"

Then somebody will ask "How big is K?" and I'll say, "K=100,000" and they'll dismiss the correlations as merely a result of "data dredging".

But if I'm making far fewer comparisons (say K=10) then people will be less likely to level that charge against my findings.

So my question is this: why are correlations acceptable in the small but not in the large? Why are the comparisons considered less valid when there are more of them? Shouldn't our inability to trust large numbers of comparisons mean we also shouldn't trust small numbers of comparisons?

• You can find the answer in the linked thread. Please read it. If you still have a question afterwards, come back here & edit your Q to state what you've learned & what you still don't understand. Then we can provide the explanation you need without just duplicating information elsewhere that already didn't help you. Dec 20, 2014 at 23:47

Why are the comparisons considered less valid when there are more of them?

For example, if you test the correlation of n uncorrelated variables, the probability that you will find at least one "significant" correlation (at the 5% level) is $1 - (1-0.05)^{\frac{n!}{(n-2)!2!}}$.