Is common to say that Machine Learning techniques represent are purely data driven methods, and them are effective only if we have a large amount of data. I focused here on supervised/predictive learning. If we intend “large” as large number of eligible predictors I agree. However some people says that also a large number of observations needed. However I’m dubious about that because one key result behind the opportunity of predictive learning is the bias-variance tradeoff (see here can help: Minimizing bias in explanatory modeling, why? (Galit Shmueli's "To Explain or to Predict") Bias/variance tradeoff tutorial Question about bias-variance tradeoff Endogeneity in forecasting ). Then, is possible to show that if the amount of observations go to infinity the tradeoff disappear and only bias become relevant. Now, In order to address the bias the theoretical knowledge about the phenomena under investigation are much more important than the computational aspects. Therefore it seems me that more observation we have more important the theory become. If what I said is correct the opposite of the underscored phrase is true: few data are good situation in predictive learning, even if less observations we have and more simple the (predictive) model should be.
My mistakes? Maybe the truth is in the middle?