What's the methodology behind the most-difference-between-groups-tag-cloud?

Question

What is the likely stats methodology used in this old OKCupid post?: http://www.economist.com/blogs/johnson/2010/10/sexuality_and_language

And this: http://blog.okcupid.com/index.php/the-real-stuff-white-people-like/

Here's the problem setup:

You have a corpus of users' self descriptions. Choose the common (issue #1!) words that maximize the difference (issue #2!) between the N groups, here N=4 groups, 2-by-2 by gender and sexuality.

So, as I understand this, you're doing a regular NLP thing, constructing a sparse matrix that has frequencies (or some kind of normed frequencies, like tf-idf). Rows are the documents, columns are counts on each of the words used across all documents within that observation. Then you use those word-occurrence-counts to construct a basis on which a set of users, say heterosexual women, has a high score, but a relatively low score on the 3 other bases chosen at the same time. The words/dimensions with the highest loading on that basis /is/ the set of words output on the chart.

Issues above that seem tricky to me:

1. What strikes people who have done this as the likely means of deciding on whether words count as "common"? Do you use thresholds? What's the appropriate way to choose most salient words? If you do something like tf-idf, pmi, this may tell you relative topicality of a document versus a corpus, but this seems to be not the same problem as the relative topicality of a set of documents versus the corpus.
2. What is the method for grouping? Since I know the number of groups, k-means seems appropriate, but how does it work mechanically on a giant sparse matrix like the word dictionaries? I more than a little hand-waved above in understanding that the word occurrence as dimensions behave like matrices I'm used to. If there are 10,000 words in the sparse matrix, aren't there an infinite number of bases? What is used to maximize the difference between groups, and is a maximum guaranteed to exist? Besides K-means, this would also naively look to me like Fisher's LDA in the sense that I'm asking for a linear combination of features (a set of words) that maximize the difference between groups.
3. bonus: If you're doing stemming to match the same word across parts of speech, what data management do you do to key the stemmed forms back to the pretty representation? (You can't invert a stem function---so f("jumping") = "jump", f("jumped") = "jump", but there is no 1-to-1 f^-1("jump") = "jumping".) Or in this case, perhaps a filter was applied at the start to isolate noun-phrases?

Any advice or especially links to code in R or Python would be helpful

Answer 1

The methodology was laid out in the book version of the blog post with some nice graphical explanations: http://www.amazon.com/Dataclysm-When-Think-Ones-Looking/dp/0385347375

In a nutshell he treated the various self-identified sub-groups as "documents" and then ranked the use of terms for the documents and the corpus. He then selected for each document top N terms ordered by the difference between document-rank and corpus-rank.