I know it's a rather debated question on Stack Exchange communities but let me explain the points of this question.

I'm writing my capstone on Machine Learning and I need to clarify deeply, giving valid references, the differences among Data Mining, Big Data, Statistical Learning and Machine Learning. As far as I understand (I'm reading The Elements of Statistical Learning, An Introduction to Statistical Learning which are well known books) and based on the questions linked below, those are basically Machine Learning. Correct me if you do not agree with me.

However, I can't find an article or book with a detailed discussion on the origins of those fields and relationship of each other and I can't cite, like, the questions below. They're not considered valid references.

1. What is the difference between data mining, statistics, machine learning and AI?

2. What is the difference between Big Data and Data Mining?

3. difference between data mining and machine learning [closed]


The issues you're noting are definitional ones where standard, widely accepted meanings for each term have yet to be agreed upon -- different authors and practitioners use them differently. I think nearly everyone would agree that there is a high degree of overlap in their use. This is frequently the case during the emergence of relatively new fields. So, 10 or more years ago, data mining was widely considered to be a "bad" thing relative to theoretical, hypothesis-driven standards of research -- the "gold standard." Today, the stigma associated with data mining has been, for the most part, removed in common parlance.

Regrettably, these considerations can devolve into dogmatic, almost religious wars of turf where the contending definitions are a function and by-product of the discipline (the "turf") within which they originate. So, machine learning has largely developed within computer science departments, whose content overlaps with statistics, but it can be treated as a wholly separate discipline from statistics with a separate literature to command. Indeed, many ML practitioners will acknowledge that their exposure and experience evolved without any statistical considerations coming into play whatsoever. A good example of this is Chen and Xie's paper on "divide and conquer" algorithms for massive data -- http://dimacs.rutgers.edu/TechnicalReports/TechReports/2012/2012-01.pdf -- which notes that D&C approaches originated in ML and computer science but without any statistical consideration given to the accuracy of the approximating results. It took statisticians like Chen and Xie to ask and address the concern.

If it's helpful, one way that might reduce some of the confusion is to think of the relationships in Boolean terms. You could even develop a text mining algorithm to show the overlap in Venn diagrams based on term usage from a set of related documents. So, AI is a subset of ML. ML is a subset of computer science that overlaps with statistics. And statistics may (or may not) be a subset of mathematics. Big data is a technical consideration that is largely a computer science concern but it is also a subset of issues having to do with IT hardware, software and data architecture. But big data has impact on statistical analysis insofar as most 20th c approaches to statistical modeling have significant "in-memory" software limitations when the data gets too big. Data mining is an approach to exploratory research that is a subset of methodological and research design issues that overlaps with statistics as well as ML. And so on.

The bottom line is that the more you read on these topics, the closer you will get to arriving at your own understanding and definitions. You may have to get creative in this regard since you are unlikely to find crisp definitions in one, two or even a few sources.

  • $\begingroup$ just for clarification of your "So, AI is a subset of ML"- AI is not a subset of ML, it is either the other way around, or there is no "subset" relation between these two. Artificial intelligence consists of many (actually - mostly) things that are not ML, such as reasoning, path search, etc. $\endgroup$ – lejlot Oct 28 '15 at 13:27
  • $\begingroup$ @lejlot Thank you for that clarification. Your point is a perfect illustration of my point regarding how little agreement there is about these terms, particularly between siloed experts of each discipline. $\endgroup$ – DJohnson Oct 28 '15 at 13:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.