4
$\begingroup$

I was going through this article to learn about how the EM algorithm can be used to use the Naive Bayes algorithm for unsupervised learning. Suppose we have the following data without labels:

1 0 1 1

0 1 1 0

0 0 1 1

We want to make 3 clusters out of them, and for a new vector x, find P(C = c | x). Now, as they suggest, we set out with assigning each vector in the training data to a class with some random probability (count). For example:

1 0 1 1 cluster = 1 count = 0.4

1 0 1 1 cluster = 2 count = 0.4

1 0 1 1 cluster = 3 count = 0.2

....

....

Now, they use the EM algorithm to update the count values, and estimate P(C), P($X_j$|C) using those values. Can someone explain to me how they are doing it? I can't comprehend the abstract language used in the article. Thanks!

$\endgroup$
1
  • $\begingroup$ The described model to me looks like Bernoulli mixture model (suggest to find references on BMM). The idea is to define a cluster as a collection of Bernoulli variables that are cluster-specific. I have the Bayesian version of this model implemented in Infer.NET. $\endgroup$ May 9, 2018 at 7:25

1 Answer 1

1
$\begingroup$

A nice Python implementation of Naive Bayes Clustering combining EM algorithm and Naive Bayes algorithm for unsupervised learning.

This implementation based on « heuristic » approximate method of moments (MOM) and tensor decomposition techniques was used to cluster high-dimensional binary data patients in groups from their electronic healthcare records. The paper « Clustering Patients with Tensor Decomposition » in Proceedings of Machine Learning for Healthcare 2017 gives detailed explanation.

$\endgroup$
1
  • $\begingroup$ Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. $\endgroup$
    – Community Bot
    Jun 18, 2023 at 7:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.