Why do Statistics, Machine learning and Operations research stand out as separate entities

Question

It seems nowadays people who work in ${\bf Statistics, \ Machine\ Learning \ and \ Operations\ research }$ all consider themselves as working in data analytics. These three categories try to solve similar problems however through different approaches. Based on my understanding the different features between statistics, machine learning and operations research is as follows:

1) ${\bf Statistics}$ provides the most mathematical rigor, however, to work under rigid mathematics, typically statisticians provide a lot of assumptions. The reason behind these assumptions is mainly to maintain interpretability and mathematical rigor. Statisticians really care about interpreting and checking the results e.g. hypothesis testing

2) The ${\bf Machine \ Learning}$ society seems to be able to work under more relaxed assumptions at the cost of interpretability. Most of machine learning methodologies are based on algorithms. These algorithms need less assumptions and seem to provide better results than similar statistics techniques, however in many cases a machine learner really cannot interpret or have a rigorous reason behind the results. It seems these algorithms are efficient however dangerous in cases when we cannot understand the underlying processes e.g. Deep learning.

3) ${\bf Operations \ Research}$ seems to be mostly focused on either Markov modeling and optimization. Markov modeling suffer from the independent increments property which is limiting in real life. While, optimization may be understood in terms of algorithms similar to ones presented in machine learning.

The above is based on my understanding, I would really appreciate feedback on the subtle differences and similarities between these three categories and why do they all stand out as separate entities

Answer 1

In machine learning "programming" = coding up an algorithm, in operations research "programming" = optimization?

More serious answer, I think the differences are more historical lineage and application area than techniques per se. One perspective on the cultures of (academic) statistics vs. machine learning I found interesting is "The Stats Handicap".

Statistics is the oldest, and originated out of mathematics and probability, perhaps emerging as a distinct discipline in the late 19th century (though much of theory is older). Of the three, statistics is perhaps the most associated with "academic science", and is certainly the most concerned with rigorous approaches to experimental design and data collection.

Operations research seems to originate closer to WWII, and is generally associated with large organizations (e.g. military, logistics/supply-chain, industrial engineering), focusing on managing and optimizing their "operations", as it were.

(In terms of "data science" traditions with a long history, another big one would be econometrics. Wikipedia says it's economics, while CV says it's statistics, for what that's worth!)

Machine learning is the most recent, but to me is more ambiguous, and at least in the popular-media it is essentially a re-branding of "AI". This broader sense includes many strands, including computer vision and probabilistic robotics. Computer science is an integral part of all of these, however.

Finally, I would say that buzzwords like "Data Science" and "Analytics" are largely marketing terms. They are less likely to be used between members of these communities, vs. when communicating with outsiders (or when outsiders are talking between themselves).

Answer 2

In my view, the differences are more cultural than methodological. All three share a common mathematical foundation in probability theory, optimization, and linear algebra. I disagree that any one of these is more "rigorous" than any other. Each field has its PhD's who do mind-bendingly rigorous and difficult research. Each also has practitioners who utilize methods and heuristics to get the job done.

As far as "analytics" there has been a concerted effort by INFORMS (the OR/MS society of the USA) to make the definition of "Analytics" more rigorous, to the point of developing a certification process (Certified Analytics Professional). The material for the exam covers far more than just statistics, machine learning, or operations research.