Finding related words

Question

I have several files, each of which contains unique terms which are related to each other(without sentence structure). So for finding the word relationships I created a dictionary of bi-grams for every two words occurring in the document, and for every match found I increment the count for the bi-gram occurrence making it more important. For example - if I have football, goal, messi. I have 6 bi-grams football-goal, goal-football, messi-football, football-messi, goal-messi, messi-goal. Doing so in a large set of document I might get several occurrences of football-goal which might tell me that these words are related. And I can further develop the co-relation with 'goal' related words. I have two questions here:

1. Are there better algorithms to achieve this?

2. If I have to use this is algorithm. What are the possible way of visualizing the co-related terms?

Answer 1

I believe that using bigrams may be a limiting choice in your case. Bigrams are a proxy to capture the context in your example and the fact that you are taking all the possible combinations will become problematic. There are various ways to deal with the issue, two of the most popular are:

1. Using your data to obtain word embeddings. In particular, Global Vectors (Glove) are built on very similar arguments with yours. Here is a very nice post that explains the rationale.
2. Use a topic model like Latent Dirichlet Allocation. In this case each of your files will be a document. LDA learns topics, that is multinomial distributions over the vocabulary of your corpus. In more simple words, topics are sets of words that are semantically similar (e.g., for a topic football the algorithm will assign high probability to words like goal, messi etc.). LDA uses exactly this assumption: words that co-occur often, belong to the same topic with high probability.

In terms of visualization, people use different ways depending on the model:

1. For embeddings, a dimensionality reduction method is used lile t-SNE and the word representations are shown in a 2-dimensional space. There you will observe that words that often co-occur are drawn close.
2. For topic models, people tend to visualize the most probable words for some topics. To see an example, check this nice post for instance where I introduce topic models.