I was going through this article to learn about how the EM algorithm can be used to use the Naive Bayes algorithm for unsupervised learning. Suppose we have the following data without labels:

1 0 1 1

0 1 1 0

0 0 1 1

We want to make 3 clusters out of them, and for a new vector x, find P(C = c | x). Now, as they suggest, we set out with assigning each vector in the training data to a class with some random probability (count). For example:

1 0 1 1 cluster = 1 count = 0.4

1 0 1 1 cluster = 2 count = 0.4

1 0 1 1 cluster = 3 count = 0.2

....

....

Now, they use the EM algorithm to update the count values, and estimate P(C), P($X_j$|C) using those values. Can someone explain to me how they are doing it? I can't comprehend the abstract language used in the article. Thanks!

  • The described model to me looks like Bernoulli mixture model (suggest to find references on BMM). The idea is to define a cluster as a collection of Bernoulli variables that are cluster-specific. I have the Bayesian version of this model implemented in Infer.NET. – Vladislavs Dovgalecs May 9 at 7:25

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.