I'm interested in learning more about nonparametric Bayesian (and related) techniques. My background is in computer science and though I have never taken a course on measure theory or probability theory, I have had a limited amount of formal training in probability and statistics. Can anyone recommend a readable introduction to these concepts to get me started?
$\begingroup$
$\endgroup$
2
-
2$\begingroup$ math.stackexchange.com might be more appropriate place to ask this, and it may already contain the answer. $\endgroup$– mpiktasJan 11, 2011 at 21:06
-
3$\begingroup$ @mpiktas Good suggestion, but bear in mind that the stated interest is technique rather than theory. Recommendations at math.SE will likely favor the latter. Moreover, you don't need to know measure theory (beyond absolute basics) to learn about NP Bayes methods, so the main focus here should be on intros to probability that focus on statistical applications. $\endgroup$– whuber ♦Jan 11, 2011 at 21:28
Add a comment
|
5 Answers
$\begingroup$
$\endgroup$
8
For a really short introduction (seven page pdf), there's also this, intended to allow you to follow papers that use a bit of measure theory :
A Measure Theory Tutorial (Measure Theory for Dummies). Maya R. Gupta. Dept of Electrical Engineering, University of Washington, 2006. (archive.org copy)
The author gives some refs at the end and says "one of the friendliest books is Resnick’s, which teaches measure theoretic graduate level probability with the assumption that you do not have a B.A. in mathematics."
S. I. Resnick, A probability path, Birkhäuser, 1999. 453 pages.
-
1$\begingroup$ Measure theory for dummies - that sounds like its written at the right level for me, I'll definitely check it out. Thanks! $\endgroup$– NickJan 27, 2011 at 6:19
-
5
-
1$\begingroup$ Eye-baling Resnick's book gives me the impression it doesn't really hold what it promises. Level of formula detail is good but lacks explanation in words for starters. $\endgroup$– tomkaJun 15, 2016 at 14:08
-
2$\begingroup$ I originally thought I was going to disagree with @tomka, but then I tried reading Resnick's book, and kind of concur :-P It threw a bunch of definitions at me, within a few pages, with no explanation. Once I had to stop and google stuff like
infinum, and limits of seuqences of infinums of sets, I tried some other options instead (currently relaly enjoying the Wernikoff, from 1957) $\endgroup$ Mar 1, 2017 at 13:18 -
$\begingroup$ @HughPerkins I tried Rosenthal's book referenced below which reads much better. $\endgroup$– tomkaMar 1, 2017 at 15:07
$\begingroup$
$\endgroup$
3
After some research, I ended up buying this when I thought I needed to know something about measure-theoretic probability:
I haven't read much of it, however, as my personal experience is in accord with Stephen Senn's quip.
-
3$\begingroup$ Despite the quip, it helps to know enough measure theory that you won't be afraid to read articles in JASA (or wherever) that could be useful or instructive. If you're going to work in stochastic processes and mess about with Ito integrals and the like, and if you care to understand the tools you'll be using, then you actually do need a serious dose of measure theory. $\endgroup$– whuber ♦Jan 12, 2011 at 15:37
-
2$\begingroup$ You're right, whuber; nevertheless I can't resist sharing another quip I've just stumbled across: "Those with a taste for foundational questions are referred to measure theory, an excursion from which few return." —James Franklin dx.doi.org/10.1007/BF02985802 $\endgroup$– onestopJan 28, 2011 at 10:16
-
1$\begingroup$ “A theoretical statistician knows all about measure theory but has never seen a measurement whereas the actual use of measure theory by the applied statistician is a set of measure zero.” $\endgroup$ Feb 27, 2017 at 11:30
$\begingroup$
$\endgroup$
1
Personally, I've found Kolmogorov's original Foundations of the Theory of Probability to be fairly readable, at least compared to most measure theory texts. Although it obviously doesn't contain any later work, it does give you an idea of most of the important concepts (sets of measure zero, conditional expectation, etc.). It is also mercifully brief, at only 84 pages.
-
3$\begingroup$ +1 for offering a classic and for the remark on brevity! $\endgroup$– whuber ♦Jan 12, 2011 at 15:37
$\begingroup$
$\endgroup$
1
Outline of Lebesgue Theory: A Heuristic Introduction by Robert E. Wernikoff. For engineers this is easily the best introduction.
-
$\begingroup$ This is highly readable, and seems to not assume I already know the stuff that I'm trying to learn :-) $\endgroup$ Mar 1, 2017 at 13:18
$\begingroup$
$\endgroup$
Jumping straight into non-parametric Bayesian analysis is quite a big first leap! Maybe get a bit of parametric Bayes under your belt first?
Three books which you may find useful from the Bayesian part of things are:
1) Probability Theory: The Logic of Science by E. T. Jaynes, Edited by G. L. Bretthorst (2003)
2) Bayesian Theory by Bernardo, J. M. and Smith, A. F. M. (1st ed 1994, 2nd ed 2007).
3) Bayesian Decision Theory J. O. Berger (1985)
A good place to see recent applications of Bayesian statistics is the FREE journal called Bayesian Analysis, with articles from 2006 to present.
answered Jan 27, 2011 at 4:35