I have data coming in the following format: UserId, GroupId, Frequency (how many times the user chose the group), Max timestamp (the last time the user chose the group).

Based on this dataset we need to figure out which groups look alike. I was thinking of using collaborative filtering but I have 3 concerns:

  1. We don't have any knowledge of "Dislikes"; we only know which groups users "Like".
  2. We don't really have ratings. In other words, users don't rate the group. Either they choose OR they don't.
  3. Frequency doesn't really imply interest level. Should I use value '1' to imply interest & '0' to imply 'don’t know the interest level'?

Should I use collaborative filtering or is there a more appropriate machine learning algorithm? Thanks for your help.

  • $\begingroup$ The title was changed by Emre. Honestly, what's more important is 'Clustering groups'. Based on frequency & date would be nice to have -:) Thanks. $\endgroup$
    – StatsFan
    Jun 1 '12 at 6:14

Yes, Collaborative Filtering is the way to go. My first approach would be to create a user-item/group matrix by just ignoring the date and using a binary preference (1, if frequency > 0, else 0, as you said).

Now given this information you can use the Jaccard Distance and a metric based cluster algorithm of your choice which can handle discrete data (k-medoids for a start) to cluster the item/group-vectors v (i.e. $v_i$ refers to whether user i likes the item/group represented by the vector or not).

Regarding the frequency

My personal experience from working with implicit preferences is, that multiple expressions does not mean anything as long as they don't cost the user anything (so just clicking into the groups indeed costs nothing, but buying from groups would). So converting the frequences to binary preferences is fine. Depending on your type of application, it may be that frequency expresses how much time the user has spent in the particular group. In this case I'd try if the usage of the pure frequencies as preferences improves the result. In this case I'd also suggest to try the influence of normalization.

Regarding "dislike or unkown ?"

In the drafted approach above you don't have to worry about the difference between "dislike" and "don't know about yet". It is enough to state that a preference of 0 indicates that the user has not yet expressed a preference yet. It is ok to assume that a user in general visits groups he/she might like more often than groups which are less interesting.

Regarding last date

This is a hard one. You may use this information to devalue older preferences, but even this is questionable. Does the user really don't like the group anymore or does he just spends his time elsewhere ? My first approach would be to create multiple clusterings for different timeframes (i.e. to model "what is hot currently") to see if differences emerge. But as I said, this is hard. You may find some inspiration Dynamic recommender systems (or in the paper linked there).

  • $\begingroup$ Thank you SOOO much Steffen for the answer. Will look at the links you recommended and will ask more questions if I have any. This is VERY useful. $\endgroup$
    – StatsFan
    Jun 1 '12 at 14:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.