I'm searching for a ready-to-use model, preferably in TensorFlow, that learns embeddings for words from a corpus, but without taking word order into account.
So far I have a
- vocabulary of 8822 words
- corpus of ~1.5M documents of variable length (mean: 11, median: 8)
The corpus reflects users and items that hold tags (from the vocabulary). My tasks is to learn embeddings of their textual descriptions. I want to build pairs, e.g. a user's tag list is [t1, t2, t3, t4, t5, t6]. But since ordering is considered irrelevant, (t1, t6) are treated equally to (t1, t2) and thus this document yields 6*5=30 positive samples (pairs).
1) How can I apply those in the context of word2vec model which require fixed window sizes?
2) After having learnt the word embeddings, can I just add them up for a given document to compare it with another, e.g. through cosine similarity?