Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I have a hierarchical feature matrix, By that I mean that each item may belongs to one or more category, so my data will look something like that

|  User |  Categ  | Item  |
|  1       |   12      | 120   |
|  2       |   15      | 411   |
|  9       |   35      | 411   |
|  1       |   15      | 321   |

where the numbers represent ids for users, categories and items respectively.

As you can see there is a hierarchy here, a set of categories for each user and a set of items within that category, where one item can belong to more than one category.

My goal is to recommend new items to the new incoming users. I already have a user similarity algorithm in place and I can find the top n similar users. Now I need to use the above table to recommend one or more items to the new user.

  • Easy way is to take the categories from the most similar user, pick some items from these categories and suggest them to the new user

But I am looking for a better approach that will consider the combination of categ-item simultaneously. This is a very large data set, so the algorithm should scale. Any suggestion?

share|improve this question
    
So a) is there something in common between item 411 in category 15 and item 411 in category 35? and b) is there anything different between the two? In the end, is it item 411 that is important rather than its category? –  Peter Ellis Dec 3 '12 at 19:58
    
for example item 411 is a nike T-Shirt. It is listed under apparel (Categ 12) and sporting goods (Categ 15). I think both categ and items should be used to recommend something otherwise we are not taking advantage of the xtra information (the categories) –  user1848018 Dec 3 '12 at 21:02

1 Answer 1

up vote 1 down vote accepted

This might be the core of a recommendation engine...so: 1. please see Machine Learning book from Manning ( a quick and dirty recommendation engine done in Python ) ...this is a nice blue print and starting point ...there is no silver bullet for recommendation engines recipes ( see Netflix case) 2. to scale up you will need to work with matrix decomposition and other linear algebra / algorithms (e.g. SVD ) or/and to work with parallel programming ( on power machines CUDA/OpenCL or in cluster e.g Hadoop)

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.