Recommendation systems as learning to rank problem

Question

Currently, I am interested in building a recommendation system. I want to build it as a learning to rank problem using either xgboost/lightgbm.

I am reading two papers about the process:

1. https://pdfs.semanticscholar.org/8f4f/d9ee2c55648a48ad571c02d821799904faa7.pdf
2. http://delivery.acm.org/10.1145/3110000/3109897/p251-freno.pdf?ip=216.240.51.5&id=3109897&acc=OPEN&key=4D4702B0C3E38B35%2E4D4702B0C3E38B35%2E4D4702B0C3E38B35%2E6D218144511F3437&CFID=808096793&CFTOKEN=20280758&acm=1505497322_2a5ed6c9c88b7a08b8d16377247cc379.

For defining the labels I plan to use an implicit score similar to the approach in the first linked paper. For each user I have access to information about whether the user purchased, liked the item, clicked on the item and also if they removed the item from the cart.

My questions are:

1. Through all of the research I have done so far the goal of the recommender system is to provide users a list of NEW content that they might be interested in buying. When training with the data I mentioned above the user has implicitly expressed interest in each item. Using the items from this list that were not purchased are not necessarily new content. I am wondering if any item the user interacted with that was not purchased or liked but interacted with should be considered as a candidate for recommendation? Has anyone actually recommended a product that a user has purchased before?

2. If indeed I am to consider only new content in the recommendation phase then am I supposed to predict a relevance score on ALL (user, item) pairs that have not yet had an interaction? What if I have over 10k products? This seems like it will not scale very well.

Any comments, insights or feedback would be greatly appreciated!

Answer 1

Although this is an old question, this might be helpful for people just starting out. Specifically the recommenderbase and nearestneighbour classes. These provide a method for calculating similarity between two items by using BM25 weighing system. BM25 is generally used in ranking webpages but the aforementioned source code modifies it for item recommendations. As the code is in Cython, it's parallelized and quite fast. The library also contains ALS(Alternating Least Squares) for recommending which I recommend you check out.

Answer 2

You could do that, depending on the size of the problem and the type of data. The Apriori algorithm comes to mind, but the complexity can get pretty rough. This would be appropriate if you were looking at sets of interactions, and wanted numbers like confidence on a given relationship. However, since you're dealing with things like “liking an item” probably not appropriate.

The fabled Top-N would be would be appropriate here if your matrix [or matrices] was sparse and binary or numeric of size users*items. If you can subset the users or item space it would make it more computationally friendly.

More or less, you would use some similarity measure like dotproduct or cosine to compute the similarity of users or items and rank those accordingly to determine likely interactions among similar users [or items].

Also, you can do the same for items, and find similar items to recommend in a serendipitous way based on whether a user “liked” a similar item, or a similar user liked a similar item, or a similar user liked that item but ultimately bought another similar item.

I'd make the assumption that removals from carts are generally motivated by either too much cost [so recommend cheaper items] or just a distinct lack of interest, so the recommendations there should be short lived temporally.

For example.

Let's assume that you have n users and m items, and you wanted to get the user similarity. You would be doing comparisons of length items, users * users-1 times. So, the worst case complexity is O(mn^2).

This isn't too bad if your data isn't hugantic ginormic [that is your users aren't huge], but you'd want to pick a similarity measure that's appropriate and quick to evaluate. E.g dotproduct for binary or cosine for numeric. There are lots of similarity measures out there [I think last I checked there were like 83 for this one algorithm] but every mathematical operation counts when you're dealing with nonlinear complexity.

Then, you'd simply say, in ranked order, your user is most likely to do next what ranked most similar user did and so on. But, if there is a temporal quality to the behavior, this won't do very well. You can still find similar users, but you'd need to account for the time variable in some clever way.

Naively, I'd just say to find another user that is at least x% similar that did a thing later and assume your user may do that. Or, use the serendipity from the set of options previously discussed.

10000 items isn't bad. It's 100 million comparisons of users length [to find similar items]. But, we should keep in mind that you only have to update as often as you'd like, and this type of matrix operation is intrinsically parallelizable.

I wrote a pretty lengthy paper on this, and it's not thoroughly clean, but you can also compress the vectors into the spectral domain to get a 4–16x speedup with a slight loss in resolution and accuracy. But for something that size, I'd break it up and grind it out the ways the smarter guys and gals did before I made a variation on the theme. The proper way sounds much better unless you're really crunched for cpu cycles.

As for dealing with the added information, it sounds fun and doable, but you'd have to make a few assumptions and rules to get it going.

Answer 3

1- One should distinguish here between two cases:

1. When the recommender system is used in production. In this case, there should be a policy to define the candidate items. For some items, you might want to recommend them although they were previously consumed, for example, ink cartridges. But in general, you have to exclude items for which there exists a clear evidence about the user's opinion towards them. Those items are used to train the recommendation model.
2. The second case is when evaluating the recommender system on an offline dataset. In this case, you want to split the items or the ratings into training and test sets. Which means, in this scenario, you would recommend items already consumed by the user just for the sake of evaluating the recommendation quality.

2- Yes, theoretically you have to rank all unconsumed items. However, you can do a lot of tricks to avoid ranking such a huge number of calculations. One idea is to apply a preprocessing step in which you identify a smaller candidate set using a ''cheap'' method. For example, items which share some attribute with the items from the user profile. Then, apply the expensive ranking operation on the smaller set of candidate items.