Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 13 down vote favorite
6

I'm interested in learning more about nonparametric Bayesian (and related) techniques. My background is in computer science and though I have never taken a course on measure theory or probability theory, I have had a limited amount of formal training in probability and statistics. Can any one recommend a readable introduction to these concepts to get me started?

share|improve this question
1  
math.stackexchange.com might be more appropriate place to ask this, and it may already contain the answer. – mpiktas Jan 11 '11 at 21:06
2  
@mpiktas Good suggestion, but bear in mind that the stated interest is technique rather than theory. Recommendations at math.SE will likely favor the latter. Moreover, you don't need to know measure theory (beyond absolute basics) to learn about NP Bayes methods, so the main focus here should be on intros to probability that focus on statistical applications. – whuber♦ Jan 11 '11 at 21:28

5 Answers

up vote 7 down vote accepted

For a really short introduction (seven page pdf), there's also this, intended to allow you to follow papers that use a bit of measure theory :

A Measure Theory Tutorial (Measure Theory for Dummies). Maya R. Gupta. Dept of Electrical Engineering, University of Washington, 2006.

He gives some refs at the end and says "one of the friendliest books is Resnick’s, which teaches measure theoretic graduate level probability with the assumption that you do not have a B.A. in mathematics."

S. I. Resnick, A probability path, Birkhäuser, 1999. 453 pages.

share|improve this answer
Measure theory for dummies - that sounds like its written at the right level for me, I'll definitely check it out. Thanks! – Nick Jan 27 '11 at 6:19
She gives ... – steadyfish Mar 22 at 17:35
up vote 6 down vote

After some research, I ended up buying this when I thought I needed to know something about measure-theoretic probability:

Jeffrey Rosenthal. A First Look at Rigorous Probability Theory. World Scientific 2007. ISBN 9789812703712.

I haven't read much of it, however, as my personal experience is in accord with Stephen Senn's quip.

share|improve this answer
+1 for the link to the blog, first citation is a gem. – mpiktas Jan 12 '11 at 8:45
1  
Despite the quip, it helps to know enough measure theory that you won't be afraid to read articles in JASA (or wherever) that could be useful or instructive. If you're going to work in stochastic processes and mess about with Ito integrals and the like, and if you care to understand the tools you'll be using, then you actually do need a serious dose of measure theory. – whuber♦ Jan 12 '11 at 15:37
You're right, whuber; nevertheless I can't resist sharing another quip I've just stumbled across: "Those with a taste for foundational questions are referred to measure theory, an excursion from which few return." —James Franklin dx.doi.org/10.1007/BF02985802 – onestop Jan 28 '11 at 10:16
up vote 1 down vote

Personally, I've found Kolmogorov's original Foundations of the Theory of Probability to be fairly readable, at least compared to most measure theory texts. Although it obviously doesn't contain any later work, it does give you an idea of most of the important concepts (sets of measure zero, conditional expectation, etc.). It is also mercifully brief, at only 84 pages.

share|improve this answer
+1 for offering a classic and for the remark on brevity! – whuber♦ Jan 12 '11 at 15:37
up vote 1 down vote

Outline of Lebesgue Theory: A Heuristic Introduction by Robert E. Wernikoff. For engineers this is easily the best introduction.

share|improve this answer
up vote 0 down vote

Jumping straight into non-parametric Bayesian analysis is quite a big first leap! Maybe get a bit of parametric Bayes under your belt first?

Three books which you may find useful from the Bayesian part of things are:

1) Probability Theory: The Logic of Science by E. T. Jaynes, Edited by G. L. Bretthorst (2003)

2) Bayesian Theory by Bernardo, J. M. and Smith, A. F. M. (1st ed 1994, 2nd ed 2007).

3) Bayesian Decision Theory J. O. Berger (1985)

A good place to see recent applications of Bayesian statistics is the FREE journal called Bayesian Analysis, with articles from 2006 to present.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.