64
$\begingroup$

I find resources like the Probability and Statistics Cookbook and The R Reference Card for Data Mining incredibly useful. They obviously serve well as references but also help me to organize my thoughts on a subject and get the lay of the land.

Q: Does anything like these resources exist for machine learning methods?

I'm imagining a reference card which for each ML method would include:

  • General properties
  • When the method works well
  • When the method does poorly
  • From which or to which other methods the method generalizes. Has it been mostly superseded?
  • Seminal papers on the method
  • Open problems associated with the method
  • Computational intensity

All these things can be found with some minimal digging through textbooks I'm sure. It would just be really convenient to have them on a few pages.

Improve this question
$\endgroup$
10
  • 5
    $\begingroup$ A nice goal, but "minimal digging through some textbooks" ? How could one even start to compress say these 20 Books for statistical learning and data mining + mloss.org/software/rating ? $\endgroup$
    – denis
    Commented Jun 27, 2011 at 10:48
  • 2
    $\begingroup$ another one: stats.stackexchange.com/questions/19311/… $\endgroup$
    – steffen
    Commented Dec 20, 2011 at 16:10
  • 2
    $\begingroup$ (+1) for the chuzpa, if such an overview would exist, I'd pay for it. The key problem is that beside some properties which could be derived from the algorithm itself, the majority of such properties or rules of the thumb is gained by experience, i.e. application. I am pretty sure a battle-hardened applied researcher or ML-framework-programmer/consultant could write something like that ... but here and now ? $\endgroup$
    – steffen
    Commented Jun 6, 2012 at 7:20
  • $\begingroup$ @Denis: the "20 books.." link does not work, can you check this? $\endgroup$
    – lmsasu
    Commented Jul 26, 2012 at 11:27
  • 7
    $\begingroup$ I'm no machine learning expert so I will defer to others to post answers but I do think that The Elements of Statistical Learning is considered a good text on the subject and is written by some of the biggest names in the field. I should add that this book is written at a high level and those I've heard recommend it did have PhDs in statistics. $\endgroup$
    – Macro
    Commented Sep 1, 2012 at 20:06

17 Answers 17

Reset to default
25
$\begingroup$

Some of the best and freely available resources are:

As to the author's question I haven't met "All in one page" solution

Improve this answer
$\endgroup$
2
  • $\begingroup$ Sergey, is Barber's book tied to Matlab ? $\endgroup$
    – denis
    Commented Jun 28, 2011 at 9:19
  • 2
    $\begingroup$ Yes, Just take a look at the book's link: The BRMLtool box is provided to help readers see how mathematical models translate into actual MAT-LAB code. $\endgroup$
    – Sergey
    Commented Jun 28, 2011 at 9:26
31
$\begingroup$

If you want to learn Machine Learning I strongly advise you enroll in the free online ML course in the winter taught by Prof. Andrew Ng.

I did the previous one in the autumn and all learning material is of exceptional quality and geared toward practical applications, and a lot easier to grok that struggling alone with a book.

It's also made a pretty low hanging fruit with good intuitive explanations and the minimum amount of math.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I just finished this course and it is awesome! Also, it gave me a great start to understanding books on machine learning. $\endgroup$
    – B Seven
    Commented Dec 20, 2011 at 17:07
  • 2
    $\begingroup$ I think this link now is coursera.org/course/ml $\endgroup$
    – n611x007
    Commented May 2, 2013 at 7:15
16
$\begingroup$

Yes, you are fine; Christopher Bishop's "Pattern Recognition and Machine Learning" is an excellent book for general reference, you can't really go wrong with it.

A fairly recent book but also very well-written and equally broad is David Barber's "Bayesian Reasoning and Machine Learning"; a book I would feel is slightly more suitable for a new-comer in the field.

I have used "The Elements of Statistical Learning" from Hastie et al. (mentioned by Macro) and while a very strong book I would not recommended it as a first reference; maybe it would serve you better as a second reference for more specialized topics. In that aspect, David MacKay's book, Information Theory, Inference, and Learning Algorithms, can also do a splendid job.

Improve this answer
$\endgroup$
4
  • 2
    $\begingroup$ +1 for Bishop. Clear development with an even level of detail. While still good I always found Hastie et al. a little bit choppy. $\endgroup$
    – conjugateprior
    Commented Sep 1, 2012 at 20:54
  • 1
    $\begingroup$ +1 -- Hastie , Tibshirani and Friedman is my personal favorite. $\endgroup$
    – StasK
    Commented Sep 1, 2012 at 23:16
  • 1
    $\begingroup$ +1 too for recommending Hastie, Tibshirani and Friedman, my personal favorite too. And thanks for the other recommendations; I'll give them a read because I really need a good book to recommend to non-statistician (or persons just entering the field). $\endgroup$
    – Néstor
    Commented Sep 2, 2012 at 20:43
  • 1
    $\begingroup$ +1 for Bishop. It's actually a great source for classical stats too, but updated and in disguise. $\endgroup$
    – conjectures
    Commented May 29, 2014 at 19:28
10
$\begingroup$

Since the consensus seems to be that this question is not a duplicate, I'd like to share my favorite for machine learner beginners:

I found Programming Collective Intelligence the easiest book for beginners, since the author Toby Segaran is is focused on allowing the median software developer to get his/her hands dirty with data hacking as fast as possible.

Typical chapter: The data problem is clearly described, followed by a rough explanation how the algorithm works and finally shows how to create some insights with just a few lines of code.

The usage of python allows one to understand everything rather fast (you do not need to know python, seriously, I did not know it before, too). DONT think that this book is only focused on creating recommender system. It also deals with text mining / spam filtering / optimization / clustering / validation etc. and hence gives you a neat overview over the basic tools of every data miner.

Improve this answer
$\endgroup$
6
$\begingroup$

Witten and Frank, "Data Mining", Elsevier 2005 is a good book for self-learning as there is a Java library of code (Weka) to go with the book and is very practically oriented. I suspect there is a more recent edition than the one I have.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ Yes, this book was to be called "Machine Learning" but the name was changed to "Data Mining" by the publishers to ride on the data mining hype at the time, nevertheless the book is about ML not DM (the two bare similarities, but are different fields!). $\endgroup$
    – clyfe
    Commented Dec 20, 2011 at 16:03
  • 1
    $\begingroup$ Tom Mitchell's book "Machine Learning" is also very good; the style is a bit old-fasioned, but the content is excellent. $\endgroup$
    – Dikran Marsupial
    Commented Dec 20, 2011 at 16:15
  • $\begingroup$ Yes, Tom Mitchell's ML is like the ML bible, really comprehensive on the field! $\endgroup$
    – clyfe
    Commented Dec 20, 2011 at 20:13
5
$\begingroup$

I have Machine Learning: An Algorithmic Perspective by Stephen Marsland and find it very useful for self-learning. Python code is given throughout the book.

I agree with what is said in this favourable review:

http://blog.rtwilson.com/review-machine-learning-an-algorithmic-perspective-by-stephen-marsland/

Improve this answer
$\endgroup$
5
$\begingroup$

enter image description here

Often the hardest part of solving a machine learning problem can be finding the right estimator for the job. Different estimators are better suited for different types of data and different problems. The flowchart below is designed to give users a bit of a rough guide on how to approach problems with regard to which estimators to try on your data. Click on any estimator in the chart below to see it’s documentation.

Improve this answer
$\endgroup$
5
$\begingroup$

"Elements of Statistical Learning" would be a great book for your purposes. The and 5th printing (2011) of the 2nd edition (2009) of the book is freely available at http://www.stanford.edu/~hastie/local.ftp/Springer/ESLII_print5.pdf

Improve this answer
$\endgroup$
5
  • 2
    $\begingroup$ it is mathematics heavy book therefore it may be difficult for self learner to follow. $\endgroup$
    – Atilla Ozgur
    Commented Dec 24, 2011 at 13:22
  • $\begingroup$ Do you know how it happens to be freely downloadable on Trevor Hastie's personnal pages when Springer charges 70$ for it? $\endgroup$
    – M. Toya
    Commented Jul 27, 2012 at 19:11
  • $\begingroup$ I don't know for sure, but I would imagine Springer wants the money, and the authors mainly want to publicize their book widely. This seems very similar to how Springer will sell you published articles while many "working paper versions" are freely available on the author's website. $\endgroup$
    – DanB
    Commented Jul 29, 2012 at 14:20
  • $\begingroup$ FYI, the download is for the 5th printing of the second edition. I love the footnote to the eipgraph "In God we trust, all others bring data" which is attributed to Deming. The footnote points out the irony that no "data" can be found confirming Deming actually saying this. $\endgroup$
    – HeatfanJohn
    Commented Sep 25, 2012 at 16:29
  • 1
    $\begingroup$ You should mention Introduction to Statistical Learning with R--it's sort of like their ESL-lite (if the math in ESL is too daunting). $\endgroup$
    – Steve S
    Commented Nov 15, 2014 at 11:19
4
$\begingroup$

The awesome-machine-learning repository seems to be a master list of resources, including code, tutorials and books.

Improve this answer
$\endgroup$
0
3
$\begingroup$

Most books mentioned in other answers are very good and you can't really go wrong with any of them. Additionally, I find the following cheat sheet for Python's scikit-learn quite useful.

Improve this answer
$\endgroup$
3
$\begingroup$

Microsoft Azure also provides a similar cheat-sheet to the scikit-learn one posted by Anton Tarasenko.

Microsoft Azure Machine Learning Algorithm Cheat Sheet

(source: https://docs.microsoft.com/en-us/azure/machine-learning/machine-learning-algorithm-cheat-sheet)

They accompany it with a notice:

The suggestions offered in this algorithm cheat sheet are approximate rules-of-thumb. Some can be bent, and some can be flagrantly violated. This is intended to suggest a starting point. (...)

Microsoft additionally provides an introductory article providing further details.

Please notice that those materials are focused on the methods implemented in Microsoft Azure.

Improve this answer
$\endgroup$
2
$\begingroup$

I like Duda, Hart and Stork "Pattern Classification". This is a recent revision of a classic text that explains everything very well. Not sure that it is updated to have much coverage of neural networks and SVMs. The book by Hastie, Tibshirani and Friedman is about the best there is but may be a bit more technical than what you are looking for and is detailed rather than an overview of the subject.

Improve this answer
$\endgroup$
2
$\begingroup$

Don't start with Elements of Statistical Learning. It is great, but it is a reference book, which doesn't sound like what you are looking for. I would start with Programming Collective Intelligence as it's an easy read.

Improve this answer
$\endgroup$
1
  • $\begingroup$ I'm not sure I would characterize ESL as a reference text. It seems more of an overview to me, i.e., you aren't going to learn the nitty gritty details of (hardly) anything. You will see the broad techniques and overarching themes. $\endgroup$
    – cardinal
    Commented Sep 3, 2012 at 14:11
2
$\begingroup$

For a first book on machine learning, which does a good job of explaining the principles, I would strongly recommend

Rogers and Girolami, A First Course in Machine Learning, (Chapman & Hall/CRC Machine Learning & Pattern Recognition), 2011.

Chris Bishop's book, or David Barber's both make good choices for a book with greater breadth, once you have a good grasp of the principles.

Improve this answer
$\endgroup$
0
$\begingroup$

I wrote a summary like that, but only on one machine learning task (Netflix Prize), and it has 195 pages: http://arek-paterek.com/book

Improve this answer
$\endgroup$
0
$\begingroup$

Check this link featuring some free ebooks on machine learning : http://designimag.com/best-free-machine-learning-ebooks/. it might be useful for you.

Improve this answer
$\endgroup$
0
$\begingroup$

A good cheatsheet is the one in Max Kuhn book Applied Predictive Modeling. In the book there is a good summary table of several ML learning models. The table is in appendix A page 549:

Table A.1: A summary of models and some of their characteristics

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.