I have seen posts about the difference between ML and Statistics. And I have also seen posts explaining that Statistical Learning is a statistical approach to ML. But then, this is confusing because what is the difference between Statistics and Statistical Learning anyways?
To finally resolve this confusion, I was hoping someone would be able to provide an answer.
Statistics is a mathematical science that studies the collection, analysis, interpretation, and presentation of data.
Statistical/Machine Learning is the application of statistical methods (mostly regression) to make predictions about unseen data. Statistical Learning and Machine Learning are broadly the same thing. The main distinction between them is in the culture.
Think of the set of questions one can ask about data as living on a simplex, where the vertices represent Confirmatory questions, Exploratory Questions, and Predictive questions about the data. Here is a visual aid I've taken from a course my supervisor has taught. Included are some questions that could be asked about data concerning how many people are in our university's gym at a given time on a given day.
One way to think about statistics vs ML is by partitioning the simplex as so
I think this is a good way to think about the difference between statistics and ML. As for statistical learning, I would put this somewhere in the purple region; methods for prediction or data mining which seem to be motivated by traditional statistical tools. The definition is highly variable and dependent on who you ask. Consequently, the distinction is of little practical importance.
Does this image clear it up?
Source: https://www.datasciencecentral.com/profiles/blogs/machine-learning-vs-statistics-in-one-picture
I have not studied this but as far as I can tell statistical learning and machine learning are the same thing.
One can make inferences and predictions in both statistical learning and inferential statistics, but the goal in statistical learning tends to be prediction over inference, whereas the reverse is true of inferential statistics.
The main difference between statistical learning and inferential statistics as I understand it is the method. In statistical learning data are split into a training set and test set, and the model learns from the training set how to maximise the accuracy of prediction of the test set. This may involve cross validation etc. In inferential statistics there is no splitting of data and training of models in any formal way.
