What are the advantages of using a Bayesian (especially a Gaussian Process method) over 'traditional' methods of classification? I understand that Gaussian process regression might be easier and more intuitive to understand as opposed to Gaussian process classification. But I am curious to know how much advantage (or mathematical insight, or computer power, or otherwise) does performing a Bayesian classification have over traditional classification algorithms like Random Forest, Logistic Regression, etc? Does it reduce the misclassification rate on the test set?
Can you kindly provide intuitive explanations, examples, links to papers or tutorials and lectures? Thank you