# Find related words

I'm trying to build a search engine that searches our products for terms to try to tag them automatically. I have a list of 100 terms that I'm trying to tag with. For example, let's say I have the word computer. I want to try to develop a method that automatically assigns the following tags:

• computer
• laptop
• desktop
• ultrabook
• chromebook

Can anyone provide information on how to start with this? This has been asked several times before, but no one has provided an answer (How to collect related words with specified one?, How to collect related words with specified one?, Finding related words).

## 2 Answers

Since, given a word $w$, you want to find related words, the best approach may be a completely ontology-based method.

I prefer this over representations over a continuous semantic space (like Word2Vec) because distributional semantics do not provide much control over how two words are similar to each other. For example, Word2Vec will often report antonyms of $w$. If you define what syntactic features to use to build these semantic representations, then this can be avoided. But then, that defeats a fundamental purpose of continuous space representations.

The idea that antonyms can be deemed similar may come as a suprise, but this does happen. The reason behind it is that these word represenations are built by looking at their contexts over a large amount of data. The main hypothesis being

words that appear in a similar context have similar meaning (a.k.a. the distributional hypothesis)

But "context" can be of two types:

1. paradigmatic, and
2. syntagmatic

Paradigmatically related words are those that can be substituted for each other in similar contexts. For example, hot-cold or boy-girl. Of course we can think of contexts where they cannot be substituted, but remember that statistical measures over large amount of data is not about possible counter-examples, but about the general picture. Pardigmatically related words usually do not appear next to each other, but appear in similar contexts -- one is present when the other is absent.

Syntagmatically related words are those that often appear together (e.g., "help-wanted").

Long story short, these two very different types of similarity cannot be distinguished by continuous space models that use only lexical contexts to build word vectors.

So how can we find related words?

We come back to the less exciting but better suited ontology-based approach. An ontology classifies words according to some semantic type. Generally, these will in turn have subtypes and supertypes. The entire structure is a giant semantic tree. The subtyping is decided by hyponymy, the "is-a" relation or meronymy, the "part-of" relation.

For general topics, Wikipedia is a rather exhaustive ontology. For example, we have the following relations:

ultrabook $\xrightarrow{is-a}$ subnotebook$\xrightarrow{is-a}$laptop

chromebook$\xrightarrow{is-a}$laptop

laptop$\xrightarrow{is-a}$personal computer$\xrightarrow{is-a}$computer

desktop$\xrightarrow{is-a}$personal computer$\xrightarrow{is-a}$computer

Since these paths are short (really, really short when you consider how large the entire semantic graph of Wikipedia is), any decent tree-based metric will detect these words as highly similar.

Unlike the distributional approach, here you have a solid reason behind why two words are similar. Recently, this approach has been used in research circles to control feature-selection in clustering and classification tasks (based on the example in your question, this paper might be of interest to you -- it presents a Wikipedia-based similarity detection for the computer networking domain, and manages to cluster terms like "Verizon" and "Time Warner Cable" together).

If you only have 100 terms to tag the easiest approach is to just assign your preferred tags to each term in a dictionary manner and then just augment the description of the item with the manually set tags.

If you are trying to do this automatically what I assume you are trying to do is augment the tags associated with a product with relevant tags, based on the words in the item description.

This is a very hard problem under active research. My first approach would be to determine similarity between words using a method such as Word2Vec, and assign to every item the top 5 most similar words according to the tags it already has. You can use the pre-trained word and phrase vectors in order to do that.

As another option you could try using Query expansion. That way you include more keywords in the user searches, and do you search against the index using the expanded query. This is the inverse from augmenting the object descriptions, and I think it might work better (less complicated, and probably less computationally intensive).

• The 100 terms is an example, I would like to expand it, but I'm at a loss of where too start. I'll check out Word2Vec, thanks! What is this type of problem called if I want to research it more and do you have any links to good papers regarding it? May 15, 2015 at 11:52
• As another option you might be better served by performing Query expansion (en.wikipedia.org/wiki/Query_expansion). I'll add this to my answer.
– Bar
May 15, 2015 at 12:58