How to work on building an engine for a website wherein we want to score/recommend stuff based on her different activities, like the music she rated or the article she read, or whether email notification is a good idea to inform her about promotions

For this as a first step, item similarity (for music and articles) has been computed via collaborative filtering, and stored. Now, as a next step, we want to be able to analyze these scenarios

  1. (A,T) : this scenario gives me a particular activity and asks me when is a user likely to do this activity.
  2. (T,A) : this scenario gives me a time slot and asks me what activity makes sense to recommend to the user at this time.
  3. (C, T) : this scenario gives me a channel (like sms, email, webpage ad, app ad) and asks which channel will the user most likely be available on right now.

As the above suggests, there are a number of modules like time, channel, interests, location, etc that can be combined interchangeably, to determine where in this kind of plane can the user be, and based on this tell whether a specific activity suggestion to her will make sense or not.

How can we go about for accomplishing this in a scalable way?

Here are a few things that I thought of, but I am not sure if they can be effective

  1. Building an item profile - create an item profile that stores time slots for the activity, channels for the activity and so on. However, this scenario is useful for (A, T) cases, but not for (T, A) cases.
  2. Treat one user as different users - we have an item similarity table, now we can store single user history as multiple users, like user1 to user1_17hrs, user1_18hrs, user1_phone, user1_tab. However, this will blow up the user data store with $2^n$ combinations.
  3. Graph DB - I have a hunch that this kind of heterogeneous data can be changed to a graph with user (user properties), channel (time as channel property) and item as nodes of graph and the edges can be rating for item, etc. Then we can use this graph db to get queries like select time and channel for user user1 and item item1, in graph db syntax. But I am not sure how to model a graph db like this.

Any leads, suggestions about my approach would be very helpful, thanks in advance!

  • $\begingroup$ You mean, you trying to come up with a feature set that will work for all three models? Just a suggestion: try starting small and modeling each one independently. I know you're worried about scaling, but it needs to work before it can scale. Also drawing small samples from your big data and developing models on those samples is a good way to get some insight that just thinking about it really hard might not get you $\endgroup$ May 19, 2015 at 12:47
  • $\begingroup$ Although on re-reading it sounds more like you're wondering what data to collect and how to store it. Can you clarify? $\endgroup$ May 19, 2015 at 12:51
  • $\begingroup$ @ssdecontrol : yes, I want to decide on a data store that answers queries like select time_slot, recommendation from table hwere user = 'user1 and channel = 'radio'. This is just an example of output i want, not looking for rdbms storage since that'll be slow. As for features to collect, I have all the event history I need, and can pick up relevant data based on which module is added to the system. $\endgroup$
    – rai
    May 20, 2015 at 7:14
  • $\begingroup$ that isn't really the question you're asking in the title and the tags aren't terribly relevant. You will need to clarify to ensure that you end up with the answers you want $\endgroup$ May 20, 2015 at 10:56
  • $\begingroup$ there is no tag for graph databases, not sure which relevant tag did i miss otherwise... $\endgroup$
    – rai
    May 20, 2015 at 12:08

1 Answer 1


Interesting question (+1). I'm not an expert in recommendation systems, so my attempt to help will be limited to emphasizing the following point (please don't ask about implementation details - you will have to figure that out by yourself or ask other people):

  • I would think that there are two somewhat different approaches: 1) one that you seem to suggest, where, if I understood correctly, you want to predict next (website) destinations of the user and generate corresponding recommendations, based on those predicted destinations; 2) another is to generate recommendations, based on tracking the user's N most recent actions (perhaps, a more dynamic option).
  • $\begingroup$ Should you ever need to visualize a graph dynamically (based on your Graph DB option), check my related answer. $\endgroup$ May 19, 2015 at 10:50
  • $\begingroup$ I didn't think of the n most recent actions, thanks, good way to think about it. $\endgroup$
    – rai
    May 20, 2015 at 7:16
  • $\begingroup$ and of course, I need to look at how to implement it. $\endgroup$
    – rai
    May 20, 2015 at 7:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.