What are the advantages of using a Bayesian (especially a Gaussian Process method) over 'traditional' methods of classification? I understand that Gaussian process regression might be easier and more intuitive to understand as opposed to Gaussian process classification. But I am curious to know how much advantage (or mathematical insight, or computer power, or otherwise) does performing a Bayesian classification have over traditional classification algorithms like Random Forest, Logistic Regression, etc? Does it reduce the misclassification rate on the test set?

Can you kindly provide intuitive explanations, examples, links to papers or tutorials and lectures? Thank you


In my opinion, a good answer is written in Pawitan (2001, p.29) "In All Likelihood: Statistical Modelling and Inference Using Likelihood." It is a better information utilization. You may consider the basic definition of a likelihood function:

$$l(\theta;x) = f(x; \theta)$$

However, this implies: $$f(\theta|x) = constant \times f(\theta)l(x|\theta)$$. It is obvious here, that classical likelihood function works if $f(\theta) = 1.$

If we do not know anything about $\theta$ prior to estimation, this approach works well. Otherwise, we need to incorporate this information into our model. Bayesians work with this particular idea and do not restrict it to 1.

However, I am not a Bayesian.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.