1
$\begingroup$

I have a data set giving the number of visits for ~20 web pages for a total of ~3000 users. To indetify "similar" users according to the number of visits of each web page, I ran a k-means clustering.

I now know which user belongs to which of the k = 3 (k is irrelevant here) clusters. But how can I characterize the clusters? Is there a way to come to a conclusion similar to "User X belongs to the cluster of users, that like web pages about News and Politics."?

$\endgroup$
1
$\begingroup$

You used a single metric to classify the users into clusters? I'll assume you have additional, descriptive information about these events. One heuristic would be to run a summary of cluster central tendencies (e.g., means, medians, etc.) based on the cluster assignments across the descriptive information. So, if you have k=3 and x=20 (both k and x are irrelevant, x being the number of descriptors or features), then the output would create a 20 (rows) by 3 (columns) summary matrix for analysis. Next, to determine how the clusters differ on each descriptor, create an index based on the cluster value divided by the global value across all users for each descriptor. This index would be like an IQ score where 100 is "normal," 120+ and 80 or less indicating descriptors that are suggesting behaviors that diverge from the norm. 120+ and 80 or less are like "quick and dirty" significance tests for between group (clusters) differences.

$\endgroup$
2
  • $\begingroup$ Just to clarify: Indeed I have one single metric, namely the count of visits of a web page. But I have this metric for 20 web pages. Besides this I only know the topic of the pages, like Politics, News, Travel, Culture... $\endgroup$ – hitchcock's_birds Dec 2 '16 at 14:22
  • $\begingroup$ Do this analysis for your single descriptor. Otherwise, your data is really impoverished. If you can get these two pieces of information, there has to be more as all of the standard website tools extract boatloads of useless metrics and information. Look into extracting more data. If somebody tells you that it doesn't exist or can't be found, they're not to be believed. $\endgroup$ – Mike Hunter Dec 2 '16 at 14:36
1
$\begingroup$

At the risk of sounding trite, you can just use the cluster centroids to characterize the clusters. The centroid of a cluster is literally the average representation of all the cluster elements. So not only are these are the natural output of k-means clustering, but they're also relatively intuitive.

What I think the answer by TrynnaDoStat is saying is that this can be risky because cluster uniformity is not a given. That is, some applications of k-means will result in clusters that are mostly homogeneous internally, and other applications will result in "messy" clusters. There are several ways to quantify the "messiness" of a clustering result: the answers to How to tell if data is "clustered" enough for clustering algorithms to produce meaningful results? contain some excellent suggestions.

$\endgroup$
0
0
$\begingroup$

This is a often a difficult task because you're trying to abstract meaning from groups that may or may not exist. When you use K-Means, you're asking the algorithm to find $K$ groups regardless if there are $K$ groups! There are also some quite restrictive assumptions on the kinds of groups you're trying to find when using K-means (How to understand the drawbacks of K-means).

Before you try and abstract meaning, I would convince yourself that you're doing a great identifying groups. This means trying different clustering algorithms and being comfortable with your selection of the number of groups while considering the possibility that there's only 1 group (see strategies here). Once you've done that, I would calculate summary statistics on each of the groups (mean vectors, variances, proportions, etc.) and see if you can abstract any meaning.

$\endgroup$
0
$\begingroup$

Use the cluster centers. They correspond to the average user behavior in each group. So treat them as if you have 3 typical users now, and study their differences.

Beware that k-means is very sensitive to noise. So if you have "one of a kind" users, they may ruin your cluster averages. K-means probably is not a very good choice because of this.

To validate your findings: 1. pay attention o cluster sizes, and 2. run k-means multiple times - if you don't get consistent findings (i.e. there is always a cluster much more interested in politics than the others) then it probably did not work well.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.