21
$\begingroup$

I'm interested in learning more about nonparametric Bayesian (and related) techniques. My background is in computer science and though I have never taken a course on measure theory or probability theory, I have had a limited amount of formal training in probability and statistics. Can anyone recommend a readable introduction to these concepts to get me started?

$\endgroup$
  • 2
    $\begingroup$ math.stackexchange.com might be more appropriate place to ask this, and it may already contain the answer. $\endgroup$ – mpiktas Jan 11 '11 at 21:06
  • 3
    $\begingroup$ @mpiktas Good suggestion, but bear in mind that the stated interest is technique rather than theory. Recommendations at math.SE will likely favor the latter. Moreover, you don't need to know measure theory (beyond absolute basics) to learn about NP Bayes methods, so the main focus here should be on intros to probability that focus on statistical applications. $\endgroup$ – whuber Jan 11 '11 at 21:28
14
$\begingroup$

For a really short introduction (seven page pdf), there's also this, intended to allow you to follow papers that use a bit of measure theory :

A Measure Theory Tutorial (Measure Theory for Dummies). Maya R. Gupta. Dept of Electrical Engineering, University of Washington, 2006. (archive.org copy)

The author gives some refs at the end and says "one of the friendliest books is Resnick’s, which teaches measure theoretic graduate level probability with the assumption that you do not have a B.A. in mathematics."

S. I. Resnick, A probability path, Birkhäuser, 1999. 453 pages.

$\endgroup$
  • 1
    $\begingroup$ Measure theory for dummies - that sounds like its written at the right level for me, I'll definitely check it out. Thanks! $\endgroup$ – Nick Jan 27 '11 at 6:19
  • 4
    $\begingroup$ She gives ... $\endgroup$ – steadyfish Mar 22 '13 at 17:35
  • $\begingroup$ Eye-baling Resnick's book gives me the impression it doesn't really hold what it promises. Level of formula detail is good but lacks explanation in words for starters. $\endgroup$ – tomka Jun 15 '16 at 14:08
  • 1
    $\begingroup$ I originally thought I was going to disagree with @tomka, but then I tried reading Resnick's book, and kind of concur :-P It threw a bunch of definitions at me, within a few pages, with no explanation. Once I had to stop and google stuff like infinum, and limits of seuqences of infinums of sets, I tried some other options instead (currently relaly enjoying the Wernikoff, from 1957) $\endgroup$ – Hugh Perkins Mar 1 '17 at 13:18
  • $\begingroup$ @HughPerkins I tried Rosenthal's book referenced below which reads much better. $\endgroup$ – tomka Mar 1 '17 at 15:07
15
$\begingroup$

After some research, I ended up buying this when I thought I needed to know something about measure-theoretic probability:

Jeffrey Rosenthal. A First Look at Rigorous Probability Theory. World Scientific 2007. ISBN 9789812703712.

I haven't read much of it, however, as my personal experience is in accord with Stephen Senn's quip.

$\endgroup$
  • 3
    $\begingroup$ Despite the quip, it helps to know enough measure theory that you won't be afraid to read articles in JASA (or wherever) that could be useful or instructive. If you're going to work in stochastic processes and mess about with Ito integrals and the like, and if you care to understand the tools you'll be using, then you actually do need a serious dose of measure theory. $\endgroup$ – whuber Jan 12 '11 at 15:37
  • $\begingroup$ You're right, whuber; nevertheless I can't resist sharing another quip I've just stumbled across: "Those with a taste for foundational questions are referred to measure theory, an excursion from which few return." —James Franklin dx.doi.org/10.1007/BF02985802 $\endgroup$ – onestop Jan 28 '11 at 10:16
  • $\begingroup$ “A theoretical statistician knows all about measure theory but has never seen a measurement whereas the actual use of measure theory by the applied statistician is a set of measure zero.” $\endgroup$ – kjetil b halvorsen Feb 27 '17 at 11:30
5
$\begingroup$

Personally, I've found Kolmogorov's original Foundations of the Theory of Probability to be fairly readable, at least compared to most measure theory texts. Although it obviously doesn't contain any later work, it does give you an idea of most of the important concepts (sets of measure zero, conditional expectation, etc.). It is also mercifully brief, at only 84 pages.

$\endgroup$
  • 2
    $\begingroup$ +1 for offering a classic and for the remark on brevity! $\endgroup$ – whuber Jan 12 '11 at 15:37
4
$\begingroup$

Outline of Lebesgue Theory: A Heuristic Introduction by Robert E. Wernikoff. For engineers this is easily the best introduction.

$\endgroup$
  • $\begingroup$ This is highly readable, and seems to not assume I already know the stuff that I'm trying to learn :-) $\endgroup$ – Hugh Perkins Mar 1 '17 at 13:18
3
$\begingroup$

Jumping straight into non-parametric Bayesian analysis is quite a big first leap! Maybe get a bit of parametric Bayes under your belt first?

Three books which you may find useful from the Bayesian part of things are:

1) Probability Theory: The Logic of Science by E. T. Jaynes, Edited by G. L. Bretthorst (2003)

2) Bayesian Theory by Bernardo, J. M. and Smith, A. F. M. (1st ed 1994, 2nd ed 2007).

3) Bayesian Decision Theory J. O. Berger (1985)

A good place to see recent applications of Bayesian statistics is the FREE journal called Bayesian Analysis, with articles from 2006 to present.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.