With item-based collaborative filtering, we utilise item ratings of similar users to a given user to generate recommendations. Research has often suggested using a hold-out test set to evaluate the algorithm e.g. 20% of data with 80% for training. However, what if in the hold-out set all ratings of a certain item are held-out? Our training data will no longer contain that item and it will never be recommended.
E.g. 5 users each view 10 films, one of which is 'Titanic'. We randomly hold-out a test set of 20% of the data per user = 2 films/user. What if 'Titanic' is in the test set for each user? It will never be recommended.