Introduction to machine learning for mathematicians In some sense this is a crosspost of mine from math.stackexchange, and I have the feeling that this site might provide a broad audience.
I am looking for a mathematical introduction to machine learning. Particularly, lots of literature that can be found is relatively imprecise and a lot of pages are spent without any content.
However, starting from such literature, I discovered the Coursera courses from Andrew Ng, the book of Bishop on pattern recognition and finally a book of Smola. Unfortunately, the book of Smola is only in draft state. In Smola's book even proofs can be found, which appeals to me. Bishop's book is already quite good, but a certain amount of rigor is missing.
In short: I am looking for a book like Smola's, that is, as precise and rigorous as possible and uses mathematical background (though short introductions are of course OK).
Any recommendations? 
 A: You will probably like Learning With Kernels by Schölkopf and Smola. Most of Schölkopf's work is mathematically rigorous. 
That said, you are probably better off reading research papers instead of textbooks. Research papers contain full derivations and proofs of convergence, bounds on performance, etc. which are very often not included in textbooks. A good place to start is the Journal of Machine Learning, which is highly regarded and fully open access. I also recommend the proceedings of conferences like ICML, NIPS, COLT and IJCNN.
A: I would suggest Understanding Machine Learning: From Theory to Algorithms by Shai Shalev-Shwartz. I admit that I read only small portions of it but I immediately noticed rigor with which author approached every problem and discussion.
A: I would recommend Elements of Statistical Learning (free PDF file). It has sufficient maths and a good introduction to all the relevant techniques - together with some insights on why the techniques work (and when they don't).
Also Introduction to Statistical Learning (which is more practical - how to do it in R). It has a course running statistical learning; you might find the lectures on YouTube (and again free PDF).
A: For what you describe, I highly recommend "Foundations of Machine Learning" by Mohri et.al.  It is an undergraduate text, but it is for really good undergraduates.  It is readable and it is the only place I have found what I would call a mathematical definition of machine learning (pac and weak pac).  It is worth reading for that reason alone.  I also have a math Phd. I'm familiar with, and like, many of the books mentioned above.  I'm particularly fond of ESL for a broad spectrum of techniques and ideas, but it's a statistics book with lots of mathematics.
