2
$\begingroup$

I run a community website with ~1600 users who made 750 000 votes on 100 000 posts. The votes are made on the Likert scale, i.e. from 1 to 5. I want to help users find like-minded people.

After some googling, I found Pearson product-moment correlation coefficient, which is apparently very easy to calculate in R.

For each pair of users, I selected votes that they made on same posts, obtaining as a result a bunch of tables like that:

user1 user2
    1     1
    5     5
    5     5
    5     1
    5     1
    1     1

Now, I can read each table as

mydata = read.table("tablename")

and run

cor(mydata[[1]],mydata[[2]])
cor.test(mydata[[1]],mydata[[2]])$p.value

to get the correlation r and significance p.

Then, I am stuck with two questions:

  1. How to properly order a list of users given r and p? Should I choose a cutoff value of p arbitrarily, then order by descending r?
  2. Did I choose the best algorithm? What are the alternatives to Pearson's r in this case?
$\endgroup$
2
  • $\begingroup$ I would like to suggest that you invert how you are thinking about this problem. Rather than starting with some mathematical formula, like the Pearson correlation, and asking how well it might work, why not think hard about what you really mean by "like-minded" and develop a quantitative way to measure that? For instance, you might decide that two users who vote identically in 99% of cases but are diametrically opposite the remaining 1% of the time aren't really like-minded at all. If you could communicate your sense of "like-minded" to us we likely could be of more help. $\endgroup$ – whuber Jan 6 '15 at 15:14
  • $\begingroup$ Well, users vote on topics ranging from breast feeding to the conflict in Ukraine. I guess that users voting the same way are like-minded, but it's just a guess. Also, most votes are either 1 or 5. That is, users either strongly like the post or strongly dislike it. Or, to be more precise, they want to push the balance on the vote towards their side. $\endgroup$ – mikhailian Jan 6 '15 at 20:17
1
$\begingroup$

This looks quite similar to the problem in the Recommender systems where based on the scores user gave to some items, new potentially interesting items are suggested.

One of the basic approaches to this problem is Collaborative filtering. From wikipedia (about user-based collaborative filtering):

Many systems can be reduced to two steps:

  • Look for users who share the same rating patterns with the active user [...]
  • Use the ratings from these like-minded users to calculate a prediction [...]

Note the first point here - it means that the system is trying to find similar users. There are many similarity functions that could be used to do this

For example,

  • Jaccard similarity $J(A,B) = {{|A \cap B|}\over{|A \cup B|}}$ - ignore the rating scores, just calculate the proportion of items both users voted for
  • Cosine similarity $\cos(\theta) = {A \cdot B \over \|A\| \|B\|} = \frac{ \sum\limits_{i=1}^{n}{A_i \cdot B_i} }{ \sqrt{\sum\limits_{i=1}^{n}{(A_i)^2}} \sqrt{\sum\limits_{i=1}^{n}{(B_i)^2}} }$ - the angle between two user vectors (in implementation it would make sense to go over only non-zero entries)
  • finally, Pearson's correlation which you already know
  • there are many more

While I didn't address some of your questions, I hope I answered the one about alternatives to Pearson's correlation coefficient.

$\endgroup$
1
  • $\begingroup$ Thanks for the ideas. I'd use Jaccard similarity, but rather on the posts both users voted for, not on the total number of posts. This is a very simple metric, great as a baseline. I haven't thought of the Cosine similarity, though. Thanks for the hint. Another hint I received offline is to view the problem through Machine Learning techniques. For instance, instead of finding similar users per user, I can search for clusters of users, with the hope to identify e.g. Young mothers, political activists, engineers, etc. $\endgroup$ – mikhailian Jan 7 '15 at 11:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.