3
$\begingroup$

In particular, I am looking for a textbook which will go over the details of derivations (including all calculus and linear algebra) for learning models and concepts such as logistic regression, Gaussian Discriminant Analysis, with full proofs for variants like Gaussian Naive Bayes.

Books such as "Elements of Statistical Learning" tend to gloss over certain details. For example, when discussing L1 Regularized Logistic Regression (Section 4.4.4), it says that "the score equations ... have the form", and then presents the form without giving the derivation.

$\endgroup$
  • $\begingroup$ You are much more likely to find the full details in publications, rather than textbooks. Very often, including all details would require more space per approach than a textbook typically has to spend. $\endgroup$ – Marc Claesen Nov 7 '13 at 19:31
  • $\begingroup$ Is there some authoritative list of publications which might contain these derivations? $\endgroup$ – merlin2011 Nov 7 '13 at 19:34
  • $\begingroup$ It's very dependent on what algorithms you are interested in. For L1 regularized LR, this might be interesting: csie.ntu.edu.tw/~cjlin/papers/l1_glmnet/long-glmnet.pdf $\endgroup$ – Marc Claesen Nov 7 '13 at 19:42
  • 2
    $\begingroup$ If a particular textbook doesn't have a derivation, it almost certainly has a citation you can reference. For instance, section 4.4.4 of ESLII (which you highlighted) contains the following references: Koh et al., 2007, Park and Hastie, 2007, Friedman et al., 2010, Efron, 1975. Check the back of the book for the full citations these references are describing. $\endgroup$ – David Marx Nov 7 '13 at 21:16
1
$\begingroup$

Have a look at 'A First Course in Machine Learning,' Simon Rogers and Mark Girolami. There are many easy to follow step by step derivations of concepts that include calculus and linear algebra. Also, you can look at google book preview to see if it fits your needs.

$\endgroup$
1
$\begingroup$

Take it as a fact that no single book can satisfy your criteria. It depends on the depth of the concepts you want to go into. It also depends on what do you mean when you say statistical concepts.

If by statistical concepts you mean distribution theory, hypothesis testing, ANOVA, Regression and other day to day statistics then Statistical Inference by Casella and Berger should be followed. If you want details of a specific advanced concept say Time Series then you can follow Time Series Analysis by Box and Jenkins for Univariate Time Series and Introduction to Multivariate Time series by Lutkepohl etc and so on for different topics

Similarly for Machine learning also . Mining of Massive datasets is a comprehensive book with proofs on Machine Learning, but again it all depends on where you are where you want to head to.

That being said, Forums,Research Papers and Publications are way to go for latest proofs, results etc.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.