People often say that Bayesian learning and maximum likelihood are two approaches used in machine learning. Main difference is that Bayesian learning tries to include the existing knowledge in the model. The results of these to approaches become closer if the prior knowledge of parameters is non informative or if the sample size approaches infinity.

In a way expectation maximization is implementation of maximum likelihood. Examples of common approach using maximum likelihood include k-means clustering and Gaussian mixture model density estimation.

What are some of the common machine learning algorithms based on Bayesian learning?


I'm not an expert in Bayesian things but I believe that the following methods are typical examples of Bayesian learning:

  • Gaussian process learning
  • Relevance vector machine
  • Hidden Markov Model (and Bayesian network learning)
  • Naive Bayes classifier (using the Laplace approximation)
  • 1
    $\begingroup$ The Naive Bayes classifier uses Bayes Rule, but isn't necessarily Bayesian in nature (the parameters can be estimated via maximum likelihood without any attempt to take into account the uncertainty in the model parameters or use of a prior distribution over parameters). Using the Laplace approximation is equivalent to a Bayesian Naive Bayes with a particular choice of prior. $\endgroup$ – Dikran Marsupial Aug 24 '15 at 9:32
  • $\begingroup$ Thanks, I was actually very unsure about that one but chose to add it nonetheless. I updated my answer with your input. $\endgroup$ – Digio Aug 24 '15 at 9:35

LDA is a very common method for unsupervised topic modeling that is Bayesian


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.