6
$\begingroup$

What is the difference between data mining and data fishing (sometimes referred to as a fishing expedition)? If there is a difference, how can you tell the one from the other? And why would one be more “valuable” for research than the other?

$\endgroup$
  • 1
    $\begingroup$ Data mining is about finding interesting patterns in a complicated dataset. Data fishing refers to the unethical activity of picking only the "correct" observations. Try "data mining" and "data dredging" articles from Wikipedia. $\endgroup$ – mmh Oct 27 '15 at 14:37
  • 1
    $\begingroup$ Fishing goes with bait, mining - with straining. $\endgroup$ – ttnphns Oct 27 '15 at 14:44
  • $\begingroup$ @mmh: what you describe is what I would call "cherry picking". Wikipedia appears to agree with my view of what data fishing is (see answer below), but personally I think that is just by chance; I don't think "data-fishing" has an exact definition and in truth I think both definitions could be considered data fishing. $\endgroup$ – Cliff AB Oct 27 '15 at 15:40
8
$\begingroup$

There is plenty of overlap between these two concepts, so there is not a clear distinction. However, I try to point out what I believe to be the differences.

In terms of statistical analysis, "a fishing expedition" just about always has a negative connotation; the idea being that the researchers started with one question about their data (i.e. "is there a linear relation between these two variables in our data?"). After coming up negative, they "recast their net" with a different question (i.e. "is there a quadratic relation between these two variables?") and so on until they finally find a "statistical significant" relation. Of course, the issue here is that the researcher did many comparisons and reported the top hit. Assuming they did not adjust their p-values for the multiple comparisons, this result will not be valid.

In contrast, with data mining (done correctly) you are starting with the understanding that you do not know which hypothesis you want to test in your data, but rather that you would like to search your data for interesting relations. As such, you will comb through your data and look for potentially interesting relations that will be reported. It is important to note that this step is really hypothesis generating, rather than confirming; to really decisively decide that the interesting relations you found in your data set are not just due to random chance, they should be confirmed in a follow up study (or moreover, independent data).

The similarities between data-fishing and data-mining is that in both cases you are inspecting a very large number of hypotheses from your data. If done correctly, data-mining is not frowned upon because it is acknowledged that you are doing this to generate interesting hypotheses to be tested later, where as data-fishing implies that the researcher did not confirm the final hypothesis they inspected in a new data set.

$\endgroup$
  • 1
    $\begingroup$ I agree: "data fishing" I have never seen in any sense but pejorative, with a limiting case in which researchers themselves wryly admit what is being done to try to disarm criticism (more often informally than in print or serious presentations), while "data mining" is a confident self-description for an activity or even a field or profession. I think "data mining" is usually coupled with insistence that it is systematic in some sense. $\endgroup$ – Nick Cox Oct 27 '15 at 17:19
  • 3
    $\begingroup$ Although it might be thought to muddy the waters, the Tukey distinction between exploratory and confirmatory is also pertinent. However, that also is hard to define systematically, as when many, many significance tests are being applied in an exploratory manner to find something interesting or useful. $\endgroup$ – Nick Cox Oct 27 '15 at 17:25
  • 2
    $\begingroup$ @NickCox: right, there's plenty of people who are proud to be professional data miners, but very few who will admit to being professional data fishers (outside the bitterest of statisticians). $\endgroup$ – Cliff AB Oct 27 '15 at 17:30
  • 1
    $\begingroup$ @Nick Interesting. It may be worth noting that Tukey did not use significance tests for data exploration. $\endgroup$ – whuber Oct 27 '15 at 18:12
  • 1
    $\begingroup$ One converse is that econometric practice pays enormous lip-service to "theory" as a guide to developing models but in practice the theory almost never implies the functional forms used and quite often "theory" just means that some variable was mentioned speculatively by a previous economist. $\endgroup$ – Nick Cox Oct 27 '15 at 18:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.