TL;DR: When implementing word co-occurence algorithms and word-embeddings, can you specify the area to be treated as the neighborhood for a word 'x' (i.e., treat a separate document as the neighborhood instead of the words near 'x')?

I have a large dataset of document pairs. For each pair, there is an itemized bill (IB) listing all materials used for a client, and there is an order report (OR) detailing how and when those materials were used. I eventually want to build a script or model that takes an IB or OR as input, and outputs a mapping (maybe a dictionary) of all terms in the IB to all descriptions in the OR, so that we can easily detail where expenses come from. Unfortunately, there are multiple ways to list an item (you might call item 'c' by the name 'a', 'x', or 'f'), and so the way an item is listed in the IB often differs from its listing in the OR.

I believe the way to tackle this is to build a word-word co-occurence matrix, where the columns are composed of the vocabulary found in all the IBs, and the rows are composed of the vocabulary of all the ORs. I can then either:

a) feed this co-occurence matrix to some word-embedding algorithm such as GLoVe or word-2-vec.

b) for a given document pair, iterate through all words (items) in the IB and use the matrix to return the most likely synonym of each of those items in the OR.

I've seen many pre-built frameworks for taking a corpus of text and doing word co-occurence based on their neighbors. Is there a way to modify these existing algorithms to consider the other document as the neighborhood rather than the words syntactically near them?

Note: I couldn't think of a better title, but I do feel like this has to be a common thing to do, i.e., cross-document co-occurence rather than within-document co-occurence.

Note 2: I have checked out HAL, KNN, GLoVe and word-2-vec, and while they all seem related to what I want to accomplish, they're either more complicated than I need (I think) or they are incompatible with the kind of synonym search I'm looking for.


1 Answer 1


There are two algorithms used for word embeddings. Skip-grams and CBOW. To answer your question directly - both allow to set 'windows-size', which is effectively what you are looking for. Depending on characteristics of your dataset, one may be better than the other. Both algorithms are really about paris of tokens, so it is really up to you how you train it and what pairs you throw at it.

Example implementation for tensorflow: https://github.com/tensorflow/tensorflow/blob/r0.8/tensorflow/examples/tutorials/word2vec/word2vec_basic.py

Having said that, I am not sure what the result is going to be in your case and if it will address your requirement. You can try feeding the algorithm with pairs of words differently, like all from IB / all from BO / pairs between IB and BO.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.