Why does Machine Learning need a lot of data while one can do statistical inference with a small set of data? I am wondering why Machine Learning needs a lot of data compared to statistical inference. In statistics, we can use a small amount of data for a statistical inference but in Machine Learning, everybody says we need a lot of data. Why does Machine Learning needs tons of more data compared to Statistics Inference?
 A: Machine learning (often) needs a lot of data because it doesn't start with a well defined model and uses (additional) data to define or improve the model. As a consequence there are often a lot of additional parameters to be estimated, parameters or settings that are already defined a-priori in non-machine-learning methods.

*

*Statistical inference, if it only requires little data, is often performed with some model that is already known/defined before the observations are made. The learning has already been done.
The goal of the inference is to estimate the few missing parameters in the model and verify the accuracy of the model.


*Machine learning is often starting with only a very minimal model or has not even a model but just a few set of rules from which a model can be created or selected.
For instance, one learns which variables are actually suitable to make good predictions or one uses a flexible neural network to come up with a function that fits well and makes good predictions.
Machine learning does not just search for a few parameters in an already fixed model. Instead it is the model itself that is being generated in machine learning. For that you need additional data.

Sometimes it is also the other way around: a lot of data needs machine learning. That is the situation with lots of variables but without a well defined model.
A: All/other things being equal (when?) machine learning models require similar quantities of data as statistical models. In general statistical models tend to have more assumptions than machine learning models and it is these additional assumptions that give you more power (assuming they are true/valid), which means that smaller samples are needed to obtain the same confidence. You can think of the difference between statistical/machine learning models as a difference between parametric and non-parametric models.
Complex models (which are more prevalent in machine learning) with many parameters do require more data (such as deep NN), but it has to do with the parameters and not the models themselves. If you built a complex statistical model with many interactions and polynomial terms you would similarly need large amounts of data to estimate all the parameters (unless you are Bayesian... then you do not even need data!).
A: A typical machine learning model contains thousands to millions of parameters, while statistical modelling is typically limited to a handful parameters.
As a rule of thumb, the minimum an amount of samples you need is proportional to the amount of parameters you want to estimate. So for statistical modelling of a handful of parameters you might only need a hundred samples, while for machine learning with millions of parameters you may need millions of samples.
A: Well, you could do inference with a small amount of data.  We just have concepts like statistical power to tell us when our results would be reliable and when they would not be.
In general, lots of data is needed in machine learning to overcome the variance in estimators/models.  Trees, as an example, are incredibly high variance estimators.  The only real way to combat that is to add more data since the variance shrinks proportional to $1/n$.
A: Machine learning does not require large amounts of data, it is just that the current bandwagon is for models that work on big data (mainly deep neural networks, which have been around since the 1990s, but before that it was SVMs and before that "shallow" neural nets), but research on other forms of machine learning has continued.  My own personal research interests are in model selection for small data, which is far from a solved problem, just not in fashion.  Another example would be Gaussian Processes, which are very good where a complex (non-linear) model is required, but the data are relatively scarce.
It is a pity that there is so much focus on deep learning and big data as it means that a lot of new practitioners are unwaware of research that was done 20 or more years ago that is still valid today, and as a result they are falling into many of the same pitfalls that we found back in the day.  Sadly ML and AI goes thorough these cycles of hype and doldrums.
At the end of the day though, ML is just statistics, but a more computationally focussed branch of statistics.
A: Machine Learning and Statistical inference deal with different type of problems and are not comparable in this point of view.
Statistical inference is used in problems that are inherently statistic, for example, if there was ten days raining then next day will more probable (using Bayesian approach) be raining as well, no need for more data.
But in machine learning, some features or patterns which exist in data must be learned. For an example in classification with machine learning, it first must learn (given a lot/enough/balanced learning data) to classify between pictures of cats and dogs, and then after learning phase, the problem in inference phase is that we show it a picture and it should tell us whether it is a cat or a dog. Now suppose that we show it 10 pictures of cats to infer and classify, and all was successfully inferred. Now, does the probability of 11th picture to be cat matter for that machine? No, because it should classify that picture based on its learned abilities to discover a cat, not the probability of being a cat after 10 cats.
