I'm building a simple recommender algorithm and as a prerequisite to calculating a prediction, I need to assign a similarity metric between each two pair of users. The next step being calculating a weighted average between all users.

My problem with Pearson's or cosine is this:
A, B are users whose vectors represent some ranking they have made on 6 items in a scale of 5.
Let A = [1,1,1,0,1,1]
Let B = [5,5,5,0,5,5]
Then the similarity s is:


So basically there's a full correlation between A,B, although user A ranks low and user B ranks high, and the users aren't literally “similar”.

I would like to use a metric that relates how 'close' A,B are to each other.
I used the following algorithm:
1. for each item i both users A,B have assigned a value r to, I used
$${{q}_{i}}=\frac{\left| {{r}_{A,i}}-{{r}_{B,i}} \right|}{scale}$$ 2.
3. Then from a scale of 0 to 1 (1 being 'alike') c(A,B) is the similarity metric (c for closeness).

I just noticed that what I did is just a variation of a normalized Minkowski Distance with a dimension of r=1

I know it's a bit general, but is the metric c makes any sense in respect to the function I want it to serve?

More to the point, will I see any improvement in the prediction if instead of doing a weighted average on let's say 10 users with the highest s to the user I try to accommodate, I'll use the ones with the highest c?


Sure, this absolutely makes sense. The only reason for defining a metric is to measure the distance between two entities in a way you want it. So you have to know what you want and then build your metric according to this.

I have one comment about your metric: Think about whether two users rating the same movies might have an influence. Now in your metric giving a 1 and a 5 is the same as having not rated at all. In your example I would say the two user are (even if they contradict in their ratings) more similar two each other than to another user who doesn't even share ratings. Think about Amazon's “users who bought this, also bought” which doesn't take ratings into account at all. I would suggest adding a term in your metric, e.g number of commonly rated items.

| cite | improve this answer | |
  • $\begingroup$ thanks. I considered your point on shared ratings, and I think it is a good idea. Like I mentioned in the post, what I did is basicly a normalized Minkowski Distance. The un-normalized formula does take into considertion the shared ratings. What I'm trying to do now is to add that consideration to the normalized metric. Do you have any ideas on how to accomplish this? $\endgroup$ – user27702 Jul 7 '13 at 21:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.