I know this is a loaded question given the infinite number of circumstances surrounding what kind of machine learning algorithm to implement. I was just wondering if there is a general framework that can hint at a situation in which "regular" machine learning algorithm will certainly outperform a neural net.

  • 3
    $\begingroup$ NN is "traditional machine learning". Do you mean to compare it with something more traditionally statistical? $\endgroup$ – Glen_b Sep 14 '17 at 2:02
  • 1
    $\begingroup$ You can't get more traditional than neural networks. Assuming that you mean deep neural nets vs any other form of ML, then the answer is still a definite yes. There's no universally best method. $\endgroup$ – Digio Sep 14 '17 at 7:26

In the case of a normally distributed response with normally distributed features, $E(y|x)$ is provably the best minimizer of mean squared predictive error.

See this question.

Or Bickel and Doksum text "Mathematical Statistics."

| cite | improve this answer | |

Not the answer you're looking for? Browse other questions tagged or ask your own question.