Unsupervised methods for Detecting Spam

What are some common unsupervised methods used to detect whether something is spam or not? For example, we may be given a large corpus of emails and need to determine whether any one of them is spam. The only problem is that we do not have any labeled training data. All we have are the emails themselves.

• If you don't know which emails are unwanted, how would you decide that since email is unwanted? – Tim Mar 21 at 18:08
• @Tim: I don't follow you. I have a collection of emails, some of which may be spam, and others ham (not spam). I don't know which ones are spam and which ones are not. – Bobby Mar 21 at 18:23
• Saying it differently: say that you have collection of emails in Chinese (assuming you don't know this language) and needed to classify them manually as spam. How would you decide? – Tim Mar 21 at 18:32
• I have a corpus of a million emails. Some of them I call "wuxles" and others I call "uxles." What advice could you--or indeed anybody--possibly provide concerning how to distinguish wuxles from uxles using some algorithm that has no information concerning which is which?? – whuber Mar 21 at 19:52

@Tim: Actually just had an idea that you could use a Hidden Markov Model? Suppose we have $$N$$ emails. In this case the vector $$\textbf{x}$$ would contain the list of emails and the vector $$\textbf{y}$$ would contain the "tags" (spam or not spam). Now we have $$p(\textbf{x}, \textbf{y}) = p(\textbf{x}| \textbf{y})p(\textbf{y})$$

$$= \prod_{i=1}^{N} p(x_i|y_i)p(y_i|y_{i-1})$$

and initialize the required probabilities (e.g. $$p(x_1| \text{spam}) = 0.5$$, etc.) and then use the Baum-Welch algorithm to update the parameters.

Or maybe we use can some other pre-trained spam model?

Added. Or maybe it should be switched around: $$p(\textbf{x}, \textbf{y}) = p(\textbf{y}| \textbf{x})p(\textbf{x})$$

$$= \prod_{i=1}^{N} p(y_i| x_i)p(x_i|x_{i-1})$$

• The question is about unsupervised learning, so there's no labels. – Tim Mar 22 at 21:21
• @Tim: That's the point of the Baum-Welch algorithm. It is an EM-algorithm. – Damien Mar 25 at 20:54
• Then "In this case the vector 𝐱 would contain the list of emails and the vector 𝐲 would contain the "tags" (spam or not spam)" is unclear. What exactly is y in this formulation? Is it observed labels or not? If it is a latent variable, then you probably need to give more detailed description of the proposed model. – Tim Mar 25 at 20:58