I know this is a loaded question given the infinite number of circumstances surrounding what kind of machine learning algorithm to implement. I was just wondering if there is a general framework that can hint at a situation in which "regular" machine learning algorithm will certainly outperform a neural net.

  • 3
    $\begingroup$ NN is "traditional machine learning". Do you mean to compare it with something more traditionally statistical? $\endgroup$ – Glen_b -Reinstate Monica Sep 14 '17 at 2:02
  • 1
    $\begingroup$ You can't get more traditional than neural networks. Assuming that you mean deep neural nets vs any other form of ML, then the answer is still a definite yes. There's no universally best method. $\endgroup$ – Digio Sep 14 '17 at 7:26

In the case of a normally distributed response with normally distributed features, $E(y|x)$ is provably the best minimizer of mean squared predictive error.

See this question.

Or Bickel and Doksum text "Mathematical Statistics."


Not the answer you're looking for? Browse other questions tagged or ask your own question.