Most efficient way to set up a questionnaire to get to know a user's taste I have a solid user-item matrix, with which I have build a collaborative filtering recommender system. I also have for each item a number of high quality features.
If a new user comes to the website (online store), I quickly want to find out his taste. Currently I do this by asking 20 questions, in which I ask him to select his most preferred product out of 4 options.
How can you use the item features and the information in the user-item matrix (and it derivaties: product popularities and latent features) to set up an efficient questionnaire to get to know the user his taste?
For example, you want to find out if the user likes popular products. So in each question with 4 options, you want to make sure there is one popular product in it, and one non popular product. But there are many features, and I am looking for a method that makes sure there is as much variation as possible in all the questions, so you learn as fast as possible.
My question is inspired by the implementation of the game 20 questions, http://www.20q.com, where you also want to ask the question that eliminates to most of the potential concepts that the user can have in mind.
Can you guide me towards some concepts or solutions?
 A: You are interested in adaptive testing. In adaptive testing scenario after answering each item from the test, the next item to be presented is choosen based on the ability of examinee as estimated based on previous items. As you can see, this is a educational/testing scenario, but the main ideas probably can be adapted to other settings as well. This kind of testing needs pretty complicated statistical methods based on Item Response Theory, so you would need to read more about those methods (disclaimer: I never worked with adaptive testing data myself). Since IRT models are very flexible by design, you can possibly easily adapt them to different scenarios.
A: You can make use of popular algorithms in two steps:


*

*Partition the space of user tastes into mutually exclusive classes. You can do it fully authomatically (e.g. do k-means clustering on the latent features), or first split the users into large meaningful classes by hand and then divide each one into subclasters by an algorithm.

*Train a decision tree on raw like/dislike data to predict these classes. Most of this data, however, would be missing. But you can use a matrix factorization algorithm (or any other appropriate model) to infer the unknown values from the known ones before building the tree. There would be some users who rated/used too few items, so this inference would be poor for them. But you may just exclude such users from this model, because they add no information anyway. 


Voila! You have the perfect questioning strategy! By design, it discriminates different tastes as fast as possible. For a new user, you just go down the decision tree asking questions "how do you like item X?", where X is the feauture used in the current node in the tree.
If the number of latent features is not too large, you can do even simpler: just train your decision tree regressor to predict all these latent features simultaneously, in a multi-output mode (scikit-learn trees certainly can do this). In this case, you don't even have to puzzle over clustering strategy, because you predict the latent features themselves instead of clusters.
