Text mining - Match product title with a description

Question

I have a file with a list of product descriptions. These product description are long strings. Eg "These blue pants are a resistant and comfortable product for tracking and ciclying. In the picture we can see the long model"

I have another file (much bigger) with product name. Eg "long sport blue pants"

I have thought about counting the number of words in the titles that contain words from the description and matching the description with the title that contains the most words. Does it make sense? Is there an algorithm for that in R? Is there any other way to do this kind of matching?

Thank you!

Answer 1

I would try to use cosine similarity between two descriptions using the Vector Space Models, and match ones with highest similarity

In R the transformation to the vector space would look like this:

library(tm)

shortDesc = all$Headline
    longDesc = all$Abstract

corpus = Corpus(VectorSource(c(shortDesc, longDesc)))
corpus = tm_map(corpus, tolower)
corpus = tm_map(corpus, PlainTextDocument)
corpus = tm_map(corpus, removePunctuation)
corpus = tm_map(corpus, removeWords, stopwords("english"))

dtm = DocumentTermMatrix(corpus)
m1 = dtm[1:100, ]
m2 = dtm[101:150, ]

Here m1 is the document matrix for the short description and m2 is the matrix with long descriptions

And now you can use cosine for matching. I don't seem to be able to find an implementation of the cosine similarity function for the tm package, but luckily it's very easy to implement this function ourselves

library(slam)

cosine = function(m1, m2=m1) {
  norm_m1 = row_sums(m1 ^ 2)
  norm_m2 = row_sums(m2 ^ 2)

  cross = as.simple_triplet_matrix(sqrt(outer(norm_m1, norm_m2)))
  tcrossprod_simple_triplet_matrix(m1, m2) / cross
}

The tm library relies on the slam package for the sparse matrices implementation, so we can use it as well and avoid converting to the dense representation. It probably should be faster this way.

Finally, let's use the cosine function for matching:

simMatrix = cosine(m1, m2)
matching = apply(simMatrix, MARGIN=1, FUN=which.max)
df = data.frame(short=shortDesc, long=longDesc[matching])

Now the dataframe df contains the short description and the best matching long description