Any statistical analysis, machine learning or data science involves some sort of optimization at the end of the day.

I'm looking for good linear and nonlinear optimization textbooks for self learning. Here are some of the criteria (subjective) for a good optimization book:

  1. Written in a didactic style or problem based learning approach with examples. if the algorithm is iterative in nature, should at least have first 2 iteration in the form of hand calculation. Theorems and poofs are great, but have having numerical examples improves learning
  2. Present optimization algorithms in Pseudo code format not as a paragraph or in words.
  3. Computer programs should accompany text in high level language such as Matlab, Mathematica or Python.
  4. Present traditional unconstrained optimization methods such as gradient descent, quasi newton, conjugate gradient, and constrained optimization such as quadratic programming, sequential quadratic programming and reduced gradient methods. Should cover both line search and trust region methods.
  5. Also present nature inspired methods such as evolutionary computing, and other techniques for nonconvex optimization since most real world problems one cannot expect convexity and often noisy, discrete and/or discontinuous.
  6. Should be written using English language and letters as opposed to Greek letters.

Any recommendations?

  • 1
    $\begingroup$ You might want to check this thread on References on numerical optimization for statisticians. That said, I do not think a book fulfilling all the requirement you list exists. Maybe K. Lange's or J. Gentle's books might be close to what you want but they do not have accompanying code nor cover evolutionary computing and that stuff... Hard ask, I am curious for any solid answers. $\endgroup$ – usεr11852 Oct 13 '18 at 23:55
  • $\begingroup$ What is wrong with Greek letters? Avoiding them in mathy texts seems quite a tough requirement. $\endgroup$ – Richard Hardy May 31 at 6:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.