I've recently came across topic known as PAC-Bayesian, but I cannot find a source to read about it. Any article that I came across are talking about its application in a specific area but there is no introduction to what it exactly is.
$\begingroup$
$\endgroup$
2
-
$\begingroup$ I'm lately interested in this topic myself, and have been looking for some good sources as well. The most interesting one I found so far is the overview/tutorial paper by David McAllester titled A PAC-Bayesian Tutorial with A Dropout Bound. $\endgroup$– sjm.majewskiJan 6, 2014 at 9:07
-
1$\begingroup$ In addition to the aforementioned materials, I recommend recent materials: 1) ICML2019 tutorial A Primer on PAC-Bayesian Learning; 2) A Primer on PAC-Bayesian Learning; 3) NIPS 2017 Workshop (Almost) 50 Shades of Bayesian Learning: PAC-Bayesian trends and insights. $\endgroup$– nzw0301Aug 7, 2019 at 16:32
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
Here are a few quick Google hits...
- PAC-Bayes Analysis: Background and Applications
- Probably Approximately Correct Learning and Vapnik-Chervonenkis Dimension
- Probably approximately correct learning on Wikipedia
- Overview of the Probably Approximately Correct (PAC) Learning Framework
From this last one, a quote:
A more refined, Bayesian extension of the PAC model is explored in [26]. Using the Bayesian approach involves assuming a prior distribution over possible target concepts as well as training instances. Given these distributions, the average error of the hypothesis as a function of training sample size, and even as a function of the particular training sample, can be defined. Also, $1 - \delta$ confidence intervals like those in the PAC model can be defined as well.
[26] $=$ W. Buntine, A Theory of Learning Classification Rules. PhD thesis, University of Technology, Sydney, 1990.
1
-
3$\begingroup$ Please don't provide link-only answers; eventually, the link will break, and then the answer will be useless. Providing a link is fine, but you should at least summarize the essentials - and provide the paper's title and author(s) - in the body of the answer. $\endgroup$– jbowmanDec 13, 2019 at 14:45
$\begingroup$
$\endgroup$
A more recent elementary introduction to PAC-Bayes
User-friendly introduction to PAC-Bayes bounds
by Pierre Alquier. It is an 80 page study of this topic.