I am starting to learn PAC learning, and have an interest in PAC-Bayes bound. However, most of the materials I found assumed binary classification only, while I am looking for the extension of PAC-Bayes bound for multiclass classification.

In chapter 3, section 3.2.2 of the textbook "Understanding machine learning: from theory to algorithms" written by Shalev-Shwartz and Ben-David, the multiclass classification is discussed and concluded with the definition of the Agnostic PAC learning for general loss function. However, I cannot find some basic extensions or proofs about multiclass classification for PAC-Bayes bound. The tutorial of PAC-Bayes bound by Langford (2005) is also about binary classification only.

I found the paper [1] interesting, but they use different performance measurement with the constrain that the minimum examples of a class $m \ge 8$. Would you like to suggest any papers or basic materials discussing PAC-Bayes bound for multiclass classification that extended some of the bounds used in binary classification such as [2]?

[1] Morvant, Emilie, Sokol Koço, and Liva Ralaivola. "PAC-Bayesian generalization bound on confusion matrix for multi-class classification." in International Conference on Machine Learning (2012).

[2] McAllester, David A. "PAC-Bayesian model averaging." Proceedings of the twelfth annual conference on Computational learning theory. ACM, 1999.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.