6
$\begingroup$

I have seen posts about the difference between ML and Statistics. And I have also seen posts explaining that Statistical Learning is a statistical approach to ML. But then, this is confusing because what is the difference between Statistics and Statistical Learning anyways?

To finally resolve this confusion, I was hoping someone would be able to provide an answer.

$\endgroup$
2
6
$\begingroup$

Statistics is a mathematical science that studies the collection, analysis, interpretation, and presentation of data.

Statistical/Machine Learning is the application of statistical methods (mostly regression) to make predictions about unseen data. Statistical Learning and Machine Learning are broadly the same thing. The main distinction between them is in the culture.

$\endgroup$
1
  • $\begingroup$ I like this answer. I know stats is different, but the difference between machine learning and statistical learning has always thrown me off. $\endgroup$ – Odisseo Jul 24 '20 at 3:31
7
$\begingroup$

Think of the set of questions one can ask about data as living on a simplex, where the vertices represent Confirmatory questions, Exploratory Questions, and Predictive questions about the data. Here is a visual aid I've taken from a course my supervisor has taught. Included are some questions that could be asked about data concerning how many people are in our university's gym at a given time on a given day.

enter image description here

One way to think about statistics vs ML is by partitioning the simplex as so

enter image description here

I think this is a good way to think about the difference between statistics and ML. As for statistical learning, I would put this somewhere in the purple region; methods for prediction or data mining which seem to be motivated by traditional statistical tools. The definition is highly variable and dependent on who you ask. Consequently, the distinction is of little practical importance.

$\endgroup$
1
  • $\begingroup$ A lot of blah blah blah on the difference between ML and Stats which we all know, but no detail on Statistical Learning which is obviously the root of the matter. $\endgroup$ – Odisseo Jul 24 '20 at 3:33
5
$\begingroup$

Does this image clear it up?

Source: https://www.datasciencecentral.com/profiles/blogs/machine-learning-vs-statistics-in-one-picture

$\endgroup$
1
  • 1
    $\begingroup$ Some people would say Machine Learning is already based on Statistics. So adding Statistics to Machine Learning = Machine Learning. $\endgroup$ – Odisseo Jul 24 '20 at 3:32
4
$\begingroup$

I have not studied this but as far as I can tell statistical learning and machine learning are the same thing.

One can make inferences and predictions in both statistical learning and inferential statistics, but the goal in statistical learning tends to be prediction over inference, whereas the reverse is true of inferential statistics.

The main difference between statistical learning and inferential statistics as I understand it is the method. In statistical learning data are split into a training set and test set, and the model learns from the training set how to maximise the accuracy of prediction of the test set. This may involve cross validation etc. In inferential statistics there is no splitting of data and training of models in any formal way.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.