# A Reference for PAC-Bayesian?

I've recently came across topic known as PAC-Bayesian, but I cannot find a source to read about it. Any article that I came across are talking about its application in a specific area but there is no introduction to what it exactly is.

Here are a few quick Google hits...

From this last one, a quote:

A more refined, Bayesian extension of the PAC model is explored in [26]. Using the Bayesian approach involves assuming a prior distribution over possible target concepts as well as training instances. Given these distributions, the average error of the hypothesis as a function of training sample size, and even as a function of the particular training sample, can be defined. Also, $1 - \delta$ confidence intervals like those in the PAC model can be defined as well.

[26] $=$ W. Buntine, A Theory of Learning Classification Rules. PhD thesis, University of Technology, Sydney, 1990.

I'm lately interested in this topic myself, and have been looking for some good sources as well. The most interesting one I found so far is the overview/tutorial paper by David McAllester titled A PAC-Bayesian Tutorial with A Dropout Bound.

This paper is a good way to start : https://arxiv.org/pdf/1901.05353.pdf

• Please don't provide link-only answers; eventually, the link will break, and then the answer will be useless. Providing a link is fine, but you should at least summarize the essentials - and provide the paper's title and author(s) - in the body of the answer. – jbowman Dec 13 '19 at 14:45