Clustering based on interactions between members of two groups

Question

I have a large dataset from a survey that describes what web pages people use. So for each person I have a list of pages that they visit and how frequently they visit them. What methods can be used to cluster both the webpages (based on similar users who use them) and the users (who use similar webpages).

Example>
User A visited web pages a, b, a, c
User B visited a, b, d, b
User C visited page e, e

Conclusion from that would be that pages a,b,c,d are somewhat similar and belong to the same cluster. The other cluster is page e. Also, users a,b are similar, the other user cluster is the one containing User C, who visits different pages.

Answer 1

If you drop the distinction between users and web pages, any of the common "strongly connected component" algorithms should work for you.

You might be able to do better by finding a similarity on users and web pages independently first, though.

Answer 2

Several thoughts. Create a variable for each web page. Each user gets 1 or 0 according as he visits the page or not. You can then do a standard hierarchical cluster on either users or pages. See hclust() and dendrogram() in base R. A heatmap() of the result might reveal interesting patterns.

An association rule could be interesting. Check out the arules package in R. It implements the apriori algorithm, amongst other things, which should give you clusters of pages that tend to be visited. arules comes with a very good explanatory manual.