Amount and sparsity of data for recommender systems

Question

I'm starting to work in a project that will have a recommender system as one of its components. I'm trying to figure out if I have the right type of data for the recommender.

The data contains ratings from implicit feedback. That is, data is either $1$ if there has been interaction (between the user and the item), or unknown if there has not been interaction. Interaction in this problem would be that the user has clicked/viewed the item.

I have about 124K users, 4,2K items, and 160K ratings. The sparsity is 99.96%.

The amount of rating per user is like this:

 Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  1.000   1.000   1.000   1.291   1.000  81.000

The amount of ratings per item is like this:

Min.  1st Qu.   Median     Mean  3rd Qu.     Max. 
    1.00     1.00     3.00    37.59    11.00 13570.00 

Clearly a majority of the users have only rated (interacted) with one item. While only a 25% of the items have a single rating.

My questions are:

1. Which levels of sparsity (amount of user-item known ratings) are typical for recommender systems?
2. How do I decrease the sparsity of my rating matrix? Should I remove users with few ratings, should I try to gather more data?

Answer 1

What you are talking about is called "cold-start problem". If you will use collaborate-filtering algorithm without any metadata, so

Which levels of sparsity (amount of user-item known ratings) are typical for recommender systems?

Generally speaking, the density 0.05% is not so bad in industrial systems.

How do I decrease the sparsity of my rating matrix? Should I remove users with few ratings, should I try to gather more data?

When you will train your model you can through the users without much data away. That makes sense, as it is hard to recommend anything specific to users without purchase history. The same for the goods that nobody buy or rated before. After this preprocessing - the sparsity of your system will decrease.

By the way, that based on the rating we can only predict ratings, but not recommendations.

Also take a look on ALS. It showed a good performance on sparse data.