Introduction to applied probability for pure mathematicians?

Question

I have a graduate-level background in pure mathematics (Measure Theory, Functional Analysis, Operator Algebra, etc.) I also have a job that requires some knowledge of probability theory (from basic principles to machine learning techniques).

My question: Can someone provide some canonical reading and reference materials that:

• Self-contained introduction to Probability theory
• Don't shy away from measure theoretic methodologies and proofs
• Provide a heavy emphasis on applied techniques.

Basically, I want a book that will teach me applied probability theory geared towards pure mathematicians. Something starting with the basic axioms of probability theory and introducing applied concepts with mathematical rigor.

As per the comments, I'll elaborate on what I need. I am doing basic-to-advanced data mining. Logistic Regression, Decision Trees, basic Stats and Probability (variance, standard deviation, likelihood, probability, likelihood, etc.), Supervised and Unsupervised machine learning (mainly clustering (K-Means, Hierarchal, SVM)).

With the above in mind, I want a book that will start at the beginning. Defining probability measures, but then also showing how those result in basic summation probabilities (which I know, intuitively, happen by integration over discrete sets). From there, it could go into: Markov Chains, Bayesian.... all the while discussing the foundational reasoning behind the theory, introducing the concepts with rigorous mathematics, but then showing how these methods are applied in the real world (specifically to data mining).

1. Does such a book or reference exist?

Thank you!

PS - I realize this is similar in scope to this question. However, I'm looking for Probability theory and not statistics (as similar as the two fields are).

Answer 1

Though I am sure that @cardinal will also put together an excellent program, let me mention a couple of books that might cover some of the things the OP is asking for.

I recently came across Probability for Statistics and Machine Learning by Anirban DasGupta, which appears to me to cover many of the probabilistic topics asked for. It is fairly mathematical in its style, though it does not seem to be "hard core" measure theoretic. The best "hard core" books are, in my opinion, Real Analysis and Probability by Dudley and Foundations of Modern Probability by Kallenberg. These two very mathematical books should be accessible given the OPs background in functional analysis and operator algebra $-$ they may even be enjoyable. Neither of them has much to say about applications though.

On the more applied side I will definitely mention Elements of Statistical Learning by Hastie et al., which provides a treatment of many modern topics and applications from statistics and machine learning. Another book that I will recommend is In All Likelihood by Pawitan. It deals with more standard statistical material and applications and is fairly mathematical too.

Answer 2

For a measure theory based introduction to probability I recommend Durrett's "Probability: Theory and Examples" (ISBN 0521765390) with Cosma Shalizi's "Almost None of the Theory of Stochastic Processes," (helpfully freely available http://www.stat.cmu.edu/~cshalizi/almost-none/v0.1.1/almost-none.pdf). I have not come across a perfect self contained book for everything after that. Some combination of MacKays's book (good for neural networks: http://www.inference.phy.cam.ac.uk/itprnn/book.html), the Koller and Friedman graphical models book (ISBN: 0262013193) and a good graduate level mathematical statistics book might work.