I read a lot of publications about data-driven modeling and machine learning. Most of them use the term interchangeably. So, is data-driven modelling and machine learning actually the same thing? If not, what are examples of data-driven models which are not considered machine learning? Or, what are examples of machine learning models which are not data-driven?
Thanks!
The term "machine learning" is somewhat a term of art, but it generally refers to the construction of algorithms that "learn through experience". The requirement of learning through experience necessitates data, and so machine learning is necessarily "data-driven" --- after all, if not from data, what else would it learn from?
When we refer to a "model" in statistics or machine learning, we really just mean a set of assumptions that describe the presumed probabilistic process for the data, and the logical consequences of the assumptions (e.g., resulting distributions of statistics, estimators, etc.). Even very broad forms of non-parametric models are considered "models", so it encompasses a lot. It is difficult to conceive of how you could generate a machine learning algorithm without some assumptions about the generative process for the data, and consequently, one can probably broadly use the term "modelling" for any machine learning process. One might quibble with this, since some machine learning algorithms are broad non-parametric methods, but even here we usually called these "models", and consequently, I think it is reasonable to say that machine learning methods are built on "models". Even such simple methods as least-squares estimation are built on underlying statistical models.
There may certainly be situations in machine learning where an algorithm is built, and even deployed, without regard to setting underlying probabilistic assumptions. If the algorithm is sufficiently adaptive (in the sense that most non-parametric models are) then one could argue that the algorithm is "model-free" insofar as it was created without regard to any model. Even then, and even if the algorithm works well in a wide class of situations, one will still tend to find that there are cases where it works well and cases where it works badly. Consequently, subsequent analysts will usually be able to figure out the kinds of assumptions required to ensure that the algorithm works well when deployed in a situation. In this case, the "modelling" gradually catches up to the initial "model-free" creation of the algorithm as we begin to learn more about the situations where the algorithm works well or badly. So you could call some machine-learning algorithms "model-free" in one sense, but modelling catches us up in the end.
In view of these considerations, I think it is reasonable to say that all machine learning involves data-driven modelling. Of course, it is possible to do data-driven modelling without using a computer algorithm at all (e.g., calculation by pen and paper), and in these cases we would not usually call that "machine learning".
TL;DR: IMO, data-driven is a broader term, but it's a matter of definition.
Different people might have different understanding of the terms "Machine Learning" and "data-driven", so I'm slightly (pleasantly) surprised that this question hasn't been closed as "opinion based". Since it still stands, I'll offer my opinion.
Historically, Machine Learning evolved as an attempt to make machines "intelligent", by allowing them to learn from "experience" (i.e. data), often by mimicking how living beings learn. So it was necessarily "data-driven". In other words, ML $\subseteq$ DD.
However, some statisticians also consider statistical modelling to be "data-driven" (e.g. Efron & Hastie, "Computer age statistical inference", p. 264). If you agree with that and if you consider data-driven statistical methods to be distinct from Machine Learning, then, obviously, "data-driven" is a broader term: DD $\supset$ ML.
(Personally, I'd rather contrast "data-driven" to "domain knowledge-driven", "probability model-based", or simply "parametric", but still leading to the same result)
There is, of course, considerable disagreement about terminology. Some statisticians consider Machine Learning to be a subset of Statistics (and most machine learners would disagree). Some machine learners consider some traditionally statistical methods, like linear or logistic regression, to be "machine learning" methods (and most statisticians would disagree). If you side with the statisticians on this point, these models would be examples of data-driven models that are not machine learning.
P.S. I disagree with bogovicj's comment. ML always builds models, only in some cases these models are not made explicit to the users. But ML algorithms certainly make some internal representations of the "things" (e.g. classes) they have learned and these representations are, for all practical purposes, synonymous to "models".
I think that the term "data-driven" has become now very popular because of deep learning techniques. The big change is that we do not longer hand-craft features, but design architectures and learning strategies that help the network learn the features directly from data. This has the advantage of being able to learn the "optimal" features for a given problem in an automated way, without having to rethink the design of those features. As an example, UNet is enormously popular architecture for segmentation. For a given problem you collect some data, choose the best fitting loss function, and you usually get pretty good results. With or without some transfer learning (fine tuning of pretrained models).
Otherwise, machine learning is about designing programs that solve problems by learning from data, instead of designing hand-crafted algorithms specific for a given situation. So yes, so data-driven modelling is just part of machine learning.
When people use data-driven modeling and machine learning interchangeably, they normally intend for them to mean the same thing. Simple definitions might be like this:
Data-driven modeling: The process of using data to derive the functional form of a model or the parameters of an algorithm.
Machine learning: The process of fitting parameters to data to minimize a cost function when the model is applied to the data. The "learning" part requires data.
One example of a data-driven modeling that is not machine learning might be physics-based modeling. In some cases of physics-based modeling, the results of an underlying physics process is compared with data, but the error with the data does not update the model parameters. Therefore, the model is not machine learning.
I don't think there are any machine learning models that are not data-driven.
One other note: These phases have no real definitions, and many people define them for their own purposes. For example, see this IBM page
