I am building a movie recommender. But unlike common recommender from movielens data, features of my movies are sets like actor list, genre list, list of producers and writers etc.

I know I can use Jaccard similarity to find similarity of 2 movies on one feature but I cant figure out how to combine these features to find similarities.

I have a thought of taking the weighted sums of features' similarity for final similarity but I cant figure out how to learn the weights.

How can I form clusters of movies? How can one improve or find a way around the suggested method above?


Primarily you would need to convert your strings into dummy variables and then compute a distance measure between rows.

The simplest of the techniques is "overlap" where you count how many common attributes are observed between a pair.

An improved version of this "goodall" weights the distance based on how infrequent a set of attributes are seen to be common.

A more exhaustive list can be seen here: Similarity Measures for Categorical Data: A Comparative Evaluation

  • $\begingroup$ Thanks! Is there any way in which I dont need to make dummy vectors? because 4 features with 1680 movies can make 6720 dummy features $\endgroup$ – Jatin Bhola Mar 22 '17 at 9:39
  • $\begingroup$ You are out of luck there. String in themselves have no meaning. $\endgroup$ – Arun Jose Mar 22 '17 at 9:40
  • $\begingroup$ get_dummies function of pandas work on dataframes in which cells have strings...but I have list of strings in my cells. Can you help with that also?That would be alot of help :-) $\endgroup$ – Jatin Bhola Mar 22 '17 at 11:49
  • $\begingroup$ If you use sparse vectors for the dummy variables, the number of features shouldn't matter much. $\endgroup$ – Anony-Mousse Mar 26 '17 at 16:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.