Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 3 down vote favorite

I'm designing a web service that will predict and recommend new items a user might like based on their expressed preferences on previous items (simple thumbs up/down interface).

I was told to look into decision trees, since they're a simple way to group things based on their characteristics, and they can be easily used to classify new items. But after having done some reading and looked at a number of examples, I'm wondering if they're the best solution in this instance.

Some problems I see:

  • There are thousands of items in the database, which means there are potentially thousands of items on which to split the tree. Running all of these splits and then calculating which ones provide the most gain (loss of entropy) seems inefficient to me.
  • The tree would be changing constantly. Most of the examples I've looked at build a tree with an initial sample set of data, and then run new items through it to classify them. While I might be able to keep a stable tree in memory after awhile, it would need to be updated pretty frequently for the first few users.
  • The tree is stored in memory, not a database. This could lead to issues if a server dies or goes offline. Also, I'm planning to user multiple machines to host the project, so there'd need to be some sort of shared memory space they could all access.
  • I don't have any categorical or descriptive data on each of the items to help with the decisions. The entire tree would have to be "did they like this other item or not?" I'm not sure if that poses any problems, but I haven't seen any other examples that use that sort of data for building the tree.

This is my first attempt at machine learning, so it could be that these are all trivial problems compared to using other solutions, and I should just stick with decision trees. But if there's another solution that seems more appropriate, or if you can address some of these concerns, I'd love to hear it!

share|improve this question
Actually the CART algorithm is the method in my opinion for constructing a classification/decision tree. It involve pruning and cross-validation which adds to the computational complexity. – Michael Chernick May 4 '12 at 16:16
In recent years in the pattern recognition/classification literature people have found that one tree or in general one classification algorithm is enough. The concept of boosting which take ensemble averages from several classifiers (or specifically trees) will do better than any single algorithm. So now people generate what Breiman called Random Forests." This adds even more complexity. Since you are dealing with large scale problems you may need a different approach. – Michael Chernick May 4 '12 at 16:16

1 Answer

up vote 3 down vote accepted

Generally, you should focus on the task, not on a method.

There are two approaches to create a recommendation engine:

  1. user-based
  2. item-based

In the former, you look for similar users (for example you group them in clusters based on demographic/sociographic attributes) and then recommend 'an item' to the specific user. [search terms: clustering]

In the latter, some patterns in combination on bought items have to be found (e.g. 90% of users who bought house bought also furnitures). Basing on that information you recommend new item (e.g. if user bought house then you should offer him/her some furnitures). [search terms: association rules]

share|improve this answer
awesome, thanks for the research tips! – Hartley Brody May 5 '12 at 19:29

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.