2
$\begingroup$

I created user sessions from server log data. Based on the URLs I categorized each server request according to the respective page content (e.g. topic_1 = main page, topic_2 = team members, etc.). The Table shows the number of times the user requested a page belonging to a topic in one session.

I want to cluster the sessions to find groups of sessions with similar interests and respectively similar browsing patterns. There are about 100 topics. The first 15 topics are requested frequently. The other topics are like a sparse matrix, but important to differentiate the sessions.

Its an exploratory data mining project. I don’t know how many interest groups there are. So I’m looking for an algorithm that doesn’t need a specified number of clusters (so no k-means or the like). The database contains about 1 million sessions. Perhaps you can suggest an implementation in R.

user_session  topic_1  topic_2  topic_3  topic_4     topic_99 topic_100
------------------------------------------------ - - ------------------
1             1        4        0        0           0        0
2             1        2        0        0           1        0
3             1        0        5        2           0        0
4             1        0        6        3           0        1
...
Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

One option: Try hierarchical clustering. e.g.

df <- read.table(header=T, text="
user_session  topic_1  topic_2  topic_3  topic_4     topic_99 topic_100
1             1        4        0        0           0        0
2             1        2        0        0           1        0
3             1        0        5        2           0        0
4             1        0        6        3           0        1")
d <- dist(df[, -1]) # '?dist' for details
hc <- hclust(d)
plot(hc)
rect.hclust(hc, k=2)

The dendrogram suggests two clusters:

enter image description here

To get them: 

cbind(session=df$user_session, cluster=cutree(hc, k=2))
#      session cluster
# [1,]       1       1
# [2,]       2       1
# [3,]       3       2
# [4,]       4       2
Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Thanks for your answer! I experimented with the distance metric and the cluster method. I think the manhattan metric works best for my high dimensional data. And I used Ward's cluttering algorithm. postimg.org/image/x2k9r3xwr $\endgroup$
    – LarsVegas
    Commented Feb 1, 2016 at 14:31
0
$\begingroup$

One way to cluster without knowing the number of clusters beforehand is the "Chinese Restaurant Process," which is described (along with an implementation in R) in this r-bloggers post.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.