I am currently trying to verify and evaluate a personalized recommender system I am working on, which seems like a huge task. Evaluating a static recommender system is rather easy and can be done with a simple bayesian framework the fast way.
But my problem lies in how to find a good way to measure the degree and quality of personalization without too much user feedback. User interaction would be OK, but the whole thing should work without the user giving much explicit feedback.
Are there any good meta-strategies to tackle this problem?
What are good values I should measure? At the moment I think about values that do not directly depend on a conscious user feedback (i.e. "satisfaction"), but on more statistically inferable factors like convergence factor of the learning algorithm, etc.
Hints anyone? Thanks guys. I don't need a full strategy, I am more hoping that anyone already found good resources how to deal with a variety of these problems, then I may just look what suits best for my specific case.