0
$\begingroup$

I have some question concern similarity measure and i need your help (I'am new in statistics)

Suppose that we have a matrix $M$ where $M(i, j)$ is the similarity measure between user $i$ and user $j$ .

Question:

  1. What similarity measure to choose and why?
  2. What is the maximum size of the matrix? What are its properties? How would you choose the store?

Thank you very much for your help bests

Improve this question
$\endgroup$
4
  • $\begingroup$ It is very difficult to know what similarity measure to choose without knowing what data you want to compare users on. There are so many options, e.g., personal characteristics (age, sex, location, interests), outputs (forum or blog posts), behaviours (click patterns, tags). You will need to be more specific. The matrix will be N x N where N is the number of users. It will be a dense symmetrical matrix (so you could store as a dense triangular matrix). $\endgroup$
    – tristan
    Commented Sep 30, 2015 at 9:41
  • $\begingroup$ Thank you tristan, Ok its a music web site each user is characterised by : id-user | country | id-artist | id-track $\endgroup$
    – user17241
    Commented Sep 30, 2015 at 9:46
  • $\begingroup$ So does this refer to the current track a user is listening to? Or are there rows for each track a user has listened to historically, so that each user appears multiple times in the database? $\endgroup$
    – tristan
    Commented Sep 30, 2015 at 9:48
  • $\begingroup$ @tristan, No, there rows for each track a user has listened to historically. $\endgroup$
    – user17241
    Commented Sep 30, 2015 at 10:07

1 Answer 1

Reset to default
1
$\begingroup$

The similarity measure you might consider choosing in this case is the Jaccard similarity index between the sets of tracks or sets of artists each user listens to.

If $X_i$ are the tracks (or artists) listened to by user $i$ and similarly $X_j$ for user $j$, the Jaccard similarity index is:

$$ M(i,j) = \frac{\left|X_i \cap X_j\right|}{\left|X_i \cup X_j\right|} $$

i.e., the ratio of the size of the intersection of tracks to the union of tracks.

So if user $i$ listens to $\{1,3,4,6,7,9\}$ and user $j$ listens to $\{2,4,5,7,8,9,10\}$ you would calculate:

\begin{align} M(i,j) &= \frac{\left|\left\{4,7,9\right\}\right|}{\left|\left\{1,2,3,4,5,6,7,8,9,10\right\}\right|} \\ &= \frac{3}{10} \end{align}

You might also consider some blending of Jaccard similarity indices for artists and tracks (e.g., weight both but weight tracks more heavily).

You will then end up with a symmetric matrix of values between 0 and 1. It will be $N\times N$ in size where $N$ is the number of users.

If users have not listened to any of the same tracks (or artists) then the index will be 0, so it may be beneficial to store in a sparse format.

You could extend this further, for example by having a measure of how similar artists are (e.g., by measuring how frequently they are both listened to by users), to allow you to connect users who listen to similar but not the same artists.

Improve this answer
$\endgroup$
2
  • $\begingroup$ last question, how determine the 20 most similar users to one user given as a parameter of a function? Than you in advance $\endgroup$
    – user17241
    Commented Oct 1, 2015 at 8:00
  • $\begingroup$ You look up that user's row in the matrix and return the columns with the 20 highest values. $\endgroup$
    – tristan
    Commented Oct 1, 2015 at 9:35

Not the answer you're looking for? Browse other questions tagged or ask your own question.