1
$\begingroup$

I'm building an e-commerce recommender system. Given a seed (bought) product I want to recommend a product that has the highest probability of being bought.

I model this as a conditional probability of recom. product (R) given the seed product (S): $P(R|S)$.

Now the problem is that purchase data is not exactly abundant. So $P(R \cap S)$ is actually quite small and I guess the results are not very statistically sound. I've decided to look at this at the category level instead.

Instead of computing $P(R|S)$ I compute $P(R|C)$ where C denotes category of the recom. product. I model user purchase decision process like: 1) user selects category 2.) user chooses the product inside category. Which, I think, could be written like

$$P(R) = P(C|S) * P(R|C)$$

where C = category of a recom. product, S = seed product, R = recom. product. I'm assuming independence of this two events.

This actually turns out to work better. It's somehow similar to taking the most probable category given the seed product and then selecting best selling product inside this category. Nevertheless I'm not happy of just blindly selecting best sellers from a category. I'm thinking how to boost relevant products i.e. include the probability from the product level $P(R|P)$ (as described in the beginning) into the formula above?

Any help greatly appreciated. I apologize for any error that occurred due to my lack of stats knowledge.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

Let me start by pointing out that there's rich literature about recommendation systems.

Does your system have the limitation of using a single seed product? If so, you can interpolate between the two modes you described: using a categorical recommendation as a back-off in the case that you don't have stronger evidence based on co-purchases.

You can model this effectively using Bayes' rule for accumulating evidence $D$:

$$P(R|D,S) \propto P(D|R,S)P(R|S)$$

where, as you wrote, $P(R|S) = \sum_{C}P(R|C)P(C|S)$. This way, for products where you have no data, you fall back to your prior. On the other hand, if you do have lots of co-purchase data for a given product, you'll be able to give more tailored recommendations.

If you're not limited to just the current product, you can do much more (think Netflix challenge). It's possible to incorporate general features about both the user (e.g. purchase history, age, gender, country, etc.) and the product (e.g. category). Here are some keywords that might be useful for you: embeddings, factorized models, collaborative filtering.

Improve this answer
$\endgroup$
7
  • $\begingroup$ Thanks a lot for the answer. In this particular case I'm limited to one seed product (think item-to-item recoms shown on the product details page). I understand that Bayes' rule is useful for refining beliefs based on the evidence. Nevertheless my knowledge is a bit lacking and I'm not sure how to use the formula you provided above. In particular how would you compute $P(D|R,S)$? If I understand the approach correctly I need to compute this in two steps. First compute probabilities on category level and then update the belief with the information I get on product level. Does that make sense? $\endgroup$
    – Domen P
    Feb 21, 2016 at 18:58
  • $\begingroup$ Sure, let's say that you have $n$ products so that your $D$ is an $n \times n$ table where, for two products $i$ and $j$, $D_{ij}$ is the number of times those two items were purchased together. You might make the assumption that $P(D|R,S)$ is just $P(D_{R,S})$. So what is the likelihood that you've seen $D_{R,S}$ co-purchases given that $R$ should be recommended for $S$? This is something you'll need to define. One very simple approach is to say that you expect to only see co-purchases of recommended products, so $P(D_{R,S}) = \frac{D_{R,S} + \epsilon}{n \epsilon + \sum_{i=1}^n D_{i,S}}$. $\endgroup$
    – Yury Sulsky
    Feb 21, 2016 at 20:46
  • $\begingroup$ (I just fixed a missing summation over $C$ in my answer). Just to expand on that last point a bit, you can think about $P(D|R,S)$ as a measure of how un-surprising it is to see $D$ in a world where $R$ is a good recommendation for $S$. This generally means that you'll expect $D_{R,S}$ to be relatively high, but you can also encode other information. For instance, if you have a similarity metric $f: n \times n \rightarrow \{0,1\}$, you might say that $P(D|R,S)=\frac{\sum_{i=1}^n f(i,R)(D_{i,S} + \epsilon)}{n\epsilon + \sum_{i=1}^n D_{i,S}}$ $\endgroup$
    – Yury Sulsky
    Feb 22, 2016 at 2:19
  • $\begingroup$ Many thanks for clarifications. I have one more noob question. In the formula $P(R|D,S) \alpha P(D|R,S)P(R|S)$ there is an $\alpha$ symbol which, I read, means that the left side is proportional to the right side. However I'm not sure how does this affect the calculation. Can we calculate the posterior simply as $P(R|D,S) = P(D|R,S)P(R|S)$? $\endgroup$
    – Domen P
    Feb 22, 2016 at 22:46
  • $\begingroup$ No problem at all. The way you read it is correct; however, the full posterior would be $P(R|D,S)=P(D|R,S)P(R|S)/P(D|S)$. If you want calibrated probabilities, you'll need to include the denominator. Equivalently, $P(D|S) = \sum_R P(D|R,S)P(R|S)$. The denominator doesn't affect which product is the best, so if you're only interested in picking the top set of recommendations, you don't need to calculate it. $\endgroup$
    – Yury Sulsky
    Feb 23, 2016 at 2:24
0
$\begingroup$

Conditional probability is a well established approach to recommendation systems. However, it is a bit flawed in logic: it proposes something to somebody because somebody else likes that thing. The more you think about it the more ridiculous it seems.

There is another approach which is based on matrix decomposition. Let's imagine that each user may be represented by a characteristic vector. It may contain, for example, a user's attributes (like age, weight, height, etc), interests (sports, literature, etc) and preferences (likes German cars, boxing, books by J.L.Borges, etc). And each product also has its characteristic vector which contains some products features (length, country of origin, relation to boxing or Latin American writers, etc).

Then, in order to understand to what extent a user i might like a product j we can just compute a vector multiplication $<U_i, P_j>$, where $U_i, P_j$ is a user vector and a product vector, consequently.

Unfortunately, the problem is that we do not know what those interests and features are. So we cannot just fill in the vectors and make some basic calculations.

We could have made a singular value decomposition. But we do not know the original matrix $[Users, Products]$, as not every user buys every product. We do know some values from that matrix, though, as some users bought some products, so we can fill it partially.

And, actually, that may be quite enough to calculate characteristic vectors for users and products. Let $L_{ij}$ be a measure of how a user i likes a product j, e.g. how many times he bought it or the amount of the product the user bought or something similar. Then obviously $L_{ij} = <U_i, P_j>$.

Since you know some of $L_{ij}$'s, you can use some gradient descent method to calculate $U_i$ and $P_j$. Notice, that $U_i, P_j$ should have the same dimensionality, i.e. the number of product features equals the number of user features. And you should decide how to choose it.

The main advantage of this approach is that you are proposing a product A to a user B because this user's features comply with the product's features (and not because another person likes the product).

On the other hand, you cannot easily interpret the model and explain (referring to common sense) why a product A is proposed to a user B as calculated features are usually impossible to interpret - these are numbers, but with a good generalization power.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.