CrossPost: https://stackoverflow.com/questions/24301743/which-machine-learning-algorithm-is-the-slowest-but-surest?noredirect=1#comment37556042_24301743

Perhaps my perception of time is augmented by the faster machine speeds these days, but I was wondering if there was a form of machine learning that takes longer but will yield drastically better results on large datasets with lots of noise. I am assuming here that faster convergence somehow has a relationship with the likelihood of becoming stuck in a local extrema. I notice little fluctuation after 100-500 epochs at which point I simply have to restart. I am currently using a feed-forward neural network for both regression and classification.

I suppose genetic algorithms seem to be the most time intensive types of "brute force" machine learning. I was also thinking that other types of neural networks could be modified (such as their momentum or learning rate to increase its range over the function). Obviously, I have tried adjusting both of these but this has not solved my problem.

  • 2
    $\begingroup$ Please do not cross-post. Decide which SE site is best suited for your question. $\endgroup$ – Momo Jun 19 '14 at 10:20

Check out the no free lunch theorem. Unfortunately, there is no one algorithm to rule them all.

That said, common methods tend to work well on a lot of problems (random forests, neural networks, support vector machines, ...).

  • $\begingroup$ The no free lunch theorem applies to the entire problem domain, correct? What about for for regression or classification? Or am I still missing something? $\endgroup$ – SilverFox Jun 19 '14 at 9:12
  • $\begingroup$ It also applies to the subdomains. The main criterion on whether or not an algorithm will perform well is whether or not its underlying assumptions are met. No approach is truly assumption-free. Take regularization for instance, which occurs in many state-of-the-art approaches. When using regularization you assume the validity of Occam's razor. $\endgroup$ – Marc Claesen Jun 19 '14 at 9:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.