Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Possible Duplicate:
The Two Cultures: statistics vs. machine learning?

What is the difference between data mining and statistical analysis?

For some background, my statistical education has been, I think, rather traditional. A specific question is posited, research is designed, and data are collected and analyzed to offer some insight on that question. As a result, I've always been skeptical of what I considered "data dredging"--looking for patterns in a large dataset and using these patterns to draw conclusions. I tend to associate the latter with data-mining and have always considered this somewhat unprincipled--along with things like algorithmic variable selection routines.

Nonetheless, there is a large and growing literature on data mining. Often I see this label referring to specific techniques--clustering, tree-based classification, etc. Yet, at least from my perspective, these techniques can be "set loose" on a set of data or used in a structured way to address a question. I'd call the former data mining and the latter statistical analysis.

I work in academic administration and have been asked to do some "data mining" to identify issues and opportunities. Consistent with my background, my first questions were--what do you want to learn and what are the things that you think contribute to issue. From their response, it was clear that me and the person asking the question had different ideas on the nature and value of data mining.

share|improve this question
1  
duplicate? stats.stackexchange.com/questions/6/… –  Neil McGuigan Aug 11 '10 at 7:26
2  
If it's duplicate I can guess that data mining and machine learning are the same thing! –  George Dontas Aug 11 '10 at 14:16
    
@George Dontas Yeah, I came here from a link in the comments if the other question in hope of seeing of there was a difference between ML and data mining. –  dg123 Feb 24 at 21:43

6 Answers 6

up vote 15 down vote accepted

Jerome Friedman wrote a paper a while back: Data Mining and Statistics: What's the Connection?, which I think you'll find interesting.

Data mining was a largely commercial concern and driven by business needs (coupled with the "need" for vendors to sell software and hardware systems to businesses). One thing Friedman noted was that all the "features" being hyped originated outside of statistics -- from algorithms and methods like neural nets to GUI driven data analysis -- and none of the traditional statistical offerings seemed to be a part of any of these systems (regression, hypothesis testing, etc). "Our core methodology has largely been ignored." It was also sold as user driven along the lines of what you noted: here's my data, here's my "business question", give me an answer.

I think Friedman was trying to provoke. He didn't think data mining had serious intellectual underpinnings where methodology was concerned, but that this would change and statisticians ought to play a part rather than ignoring it.

My own impression is that this has more or less happened. The lines have been blurred. Statisticians now publish in data mining journals. Data miners these days seem to have some sort of statistical training. While data mining packages still don't hype generalized linear models, logistic regression is well known among the analysts -- in addition to clustering and neural nets. Optimal experimental design may not be part of the data mining core, but the software can be coaxed to spit out p-values. Progress!

share|improve this answer
    
This is a great paper and is consistent with my perspective on what data mining is and how it differs from statistics. The catch is, it's from 1997! Note an indictment of the paper or your recommendation, but the degree to which I have kept up with data mining. It sounds like I need to grab a current book on data mining to catch up. –  Brett Magill Aug 11 '10 at 14:19
    
Heh, I kept the date out on purpose because I thought it would be amusing to notice the time span. :) The books by Michael Berry and Gordon Linoff are pretty good and will appeal to statisticians (for the broader exposure rather than learning statistical techniques). If you want a sense of the fuzzy, "enterprise" side of this field, skimming through one of the books on a vendor product, like SAS's Enterprise Miner or SPSS's Clementine, may help. I wouldn't recommend buying them unless you're going to work with the product itself. –  ars Aug 11 '10 at 17:50

The difference between statistics and data mining is largely a historical one, since they came from different traditions: statistics and computer science. Data mining grew in parallel out of work in the area of artificial intelligence and statistics.

Section 1.4 from Witten & Frank summarizes my viewpoint so I'm going to quote it at length:

What's the difference between machine learning and statistics? Cynics, looking wryly at the explosion of commercial interest (and hype) in this area, equate data mining to statistics plus marketing. In truth, you should not look for a dividing line between machine learning and statistics because there is a continuum--and a multidimensional one at that--of data analysis techniques. Some derive from the skills taught in standard statistics courses, and others are more closely associated with the kind of machine learning that has arisen out of computer science. Historically, the two sides have had rather different traditions. If forced to point to a single difference of emphasis, it might be that statistics has been more concerned with testing hypotheses, whereas machine learning has been more concerned with formulating the process of generalization as a search through possible hypotheses...

In the past, very similar methods have developed in parallel in machine learning and statistics...

But now the two perspectives have converged.

N.B.1 IMO, data mining and machine learning are very closely related terms. In one sense, machine learning techniques are used in data mining. I regularly see these terms as interchangeable, and in so far as they are different, they usually go together. I would suggest looking through "The Two Cultures" paper as well as the other threads from my original question.

N.B.2 The term "data mining" can have a negative connotation when used colloquially to mean letting some algorithm loose on the data without any conceptual understanding. The sense is that data mining will lead to spurious results and over-fitting. I typically avoid using the term when talking to non-experts as a result, and instead use machine learning or statistical learning as a synonym.

share|improve this answer
    
About N.B.2 - I think you're exactly right regarding the connotation of data mining and I had not made the connection to machine learning. My training always emphasized the problems of over-fitting, spuriousness, and capitalizing on chance and as such I have been skeptical on DM--and still am, perhaps until someone actually tells me WHAT they are doing and HOW. Thanks. –  Brett Magill Aug 11 '10 at 16:05
1  
My only quibble on the ML/DM distinction would be that I think DM is broader. For example, OLAP and related tools include mining technologies. But these come from the database side of computer science rather than machine learning. The role of commerce in shaping the "meaning" of data mining is hard to ignore -- it brings in elements of management sciences, operations research, machine learning and statistics as required. It also gives the impression of something flimsy, but that's usually a problem for purists not practitioners. –  ars Aug 11 '10 at 16:47
    
@ars: I agree. I was trying to say that a little by saying "machine learning techniques are used in data mining" (i.e. data mining is a super-set). Your point about the commercial applications is also spot on. Although someone in a commercial application now-a-days might refer to their work as something else (e.g. "data science"). –  Shane Aug 11 '10 at 19:36
    
Right, I should have said I was trying to flesh out the differences, rather than actually quibble with what you wrote. Apologies for the misdirection. Good point on changing times and terms like the adoption of "data science". Doesn't one of Gelman's books start with something like "statistics is the science of data"? So "they're" stealing from statisticians. Again. :) –  ars Aug 11 '10 at 20:39

Data mining is categorized as either Descriptive or Predictive. Descriptive data mining is to search massive data sets and discover the locations of unexpected structures or relationships, patterns, trends, clusters, and outliers in the data. On the other hand, Predictive is to build models and procedures for regression, classification, pattern recognition, or machine learning tasks, and assess the predictive accuracy of those models and procedures when applied to fresh data.

The mechanism used to search for patterns or structure in high-dimensional data might be manual or automated; searching might require interactively querying a database management system, or it might entail using visualization software to spot anomalies in the data. In machine-learning terms, descriptive data mining is known as unsupervised learning, whereas predictive data mining is known as supervised learning.

Most of the methods used in data mining are related to methods developed in statistics and machine learning. Foremost among those methods are the general topics of regression, classification, clustering, and visualization. Because of the enormous sizes of the data sets, many applications of data mining focus on dimensionality-reduction techniques (e.g., variable selection) and situations in which high-dimensional data are suspected of lying on lower-dimensional hyperplanes. Recent attention has been directed to methods of identifying high-dimensional data lying on nonlinear surfaces or manifolds.

There are also situations in data mining when statistical inference — in its classical sense — either has no meaning or is of dubious validity: the former occurs when we have the entire population to search for answers, and the latter occurs when a data set is a “convenience” sample rather than being a random sample drawn from some large population. When data are collected through time (e.g., retail transactions, stock-market transactions, patient records, weather records), sampling also may not make sense; the time-ordering of the observations is crucial to understanding the phenomenon generating the data, and to treat the observations as independent when they may be highly correlated will provide biased results.

The central components of data mining are — in addition to statistical theory and methods — computing and computational efficiency, automatic data processing, dynamic and interactive data visualization techniques, and algorithm development.

One of the most important issues in data mining is the computational problem of scalability. Algorithms developed for computing standard exploratory and confirmatory statistical methods were designed to be fast and computationally efficient when applied to small and medium-sized data sets; yet, it has been shown that most of these algorithms are not up to the challenge of handling huge data sets. As data sets grow, many existing algorithms demonstrate a tendency to slow down dramatically (or even grind to a halt).

share|improve this answer

I previously wrote a post where I made a few observations comparing data mining to psychology. I think these observations may capture some of the differences you are identifying:

  1. "Data mining seems more concerned with prediction using observed variables than with understanding the causal system of latent variables; psychology is typically more concerned with the causal system of latent variables.
  2. Data mining typically involves massive datasets (e.g. 10,000 + rows) collected for a purpose other than the purpose of the data mining. Psychological datasets are typically small (e.g., less than 1,000 or 100 rows) and collected explicitly to explore a research question.
  3. Psychological analysis typically involves testing specific models. Automated model development approaches tend not to be theoretically interesting." - Data Mining and R
share|improve this answer
    
I think points 2 and 3 are useful comments and consistent with what I see as the distinction between the two SA and DM. I'm not so sure about your first point. I've done statistical work where I was interested in improving understanding about causal relationships. However, I've also done statistical work where the task was to take known relationships and develop models with the sole purpose of prediction but which shared non of the other features of "data mining". –  Brett Magill Aug 11 '10 at 14:39

This might be a duplicate, but I am going to answer it anyways.

Data mining is statistics, with some minor differences. You can think of it as re-branding statistics, because statisticians are kinda weird.

It is often associated with Computational Statistics, ie only stuff you can do with a computer.

Data miners stole a significant proportion of multivariate statistics and called it their own. Check the table of contents of any 1990s multivariate book and compare it to a new data mining book. Very similar.

Statistics is associated with testing hypotheses and with model building, whereas Data Mining is more associated with prediction and classification, regardless of whether there is an understandable model.

share|improve this answer
1  
What's the duplicate? I can't see anything obvious. –  Rob Hyndman Aug 11 '10 at 6:43
1  
Pretty similar to this one I thought: stats.stackexchange.com/questions/6/… –  Neil McGuigan Aug 11 '10 at 7:25
1  
ok. I was searching for data mining, not machine learning. Please vote to close if you think it is a duplicate. –  Rob Hyndman Aug 11 '10 at 7:36
    
Hmm, so Data Mining == Machine Learning? –  ars Aug 11 '10 at 10:47
1  
1) I'm not seeing the comp stat distinction. There's not much that statisticians do that doesn't require a computer. I suppose you mean computationally intensive procedures such as iterative solutions, etc? But then, these are also common in modern statistical work that is not data mining. 2) In my own (stats) work, I've been interested in model building for explanation and for prediction, depending on the problem-I wouldn't have considered that data mining. 3) I'm left with the conclusion that modern DM is a particular application of statistics, which I think is a fine conclusion. –  Brett Magill Aug 11 '10 at 14:44

I don't think the distinction you make is really related to the difference between data mining and statistical analysis. You are talking about the difference between exploratory analysis and modelling-prediction approach.

I think the tradition of statisic is build with all steps : exploratory analysis, then modeling, then estimation, then testing, then forecasting/infering. Statistician do exploratory analysis to figure out what the data looks like (function summary under R !) I guess datamining is less structured and could be identified with exploratory analysis. However it uses techniques from statistics that are from estimation, forecasting, classification ....

share|improve this answer
    
I can buy that. Data mining is more exploratory application of statistical techniques. Though, I don't think that distinction is enough. When I'm doing EDA on my set of 100 observations from a designed experiment, I don't think anyone would call that data mining, would they? –  Brett Magill Aug 11 '10 at 14:42

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.