I know that Newton Raphson, Expectation & Maximization, and Gradient Descent are all known to be optimization methods. Somehow, I wonder why Gradient Descent is chosen to be used in most of Machine Learning applications but I never heard that Expectation & Maximization or Newton Raphson algorithms have been applied. Hope to hear some.

  • 1
    $\begingroup$ How do you define "Machine learning"? If broadly, then they are all used... if narrowly, not every optimization algorithm is "best" for every type of problem, e.g., Random Forests will use different algorithms than maximum likelihood estimation of the parameters of a t-distribution. $\endgroup$ – jbowman Dec 31 '19 at 1:47
  • $\begingroup$ Yes, you make sense. Let's say Machine Learning applications such as Linear Regression, Logistic Regression, and Softmax Regression. $\endgroup$ – Changhee Kang Dec 31 '19 at 1:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.