4
$\begingroup$

I want to capture relationships between high-level concepts, as opposed to just between words, using word2vec or similar approach. What would be a promising way to achieve that? I was thinking of modifying the objective function to use something else that word co-occurrence counts, is there any better approach?

A bit of background: at present, I'm using Siamese LSTMs to classify whether a title-reference pair from a systematic review is relevant. Currently, the inputs are word vectors generated using fastText.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

capture relationships between high-level concepts

It's not clear what you mean by this. Do you have these relationships or do you want to learn them (from what?)?

If you mean encoding using existing structure, then you can use node2vec on your knowledge base (for example WordNet, you can also search for ontology/taxonomy).

Poincare embeddings also come to mind, but they encode points in hyperbolic space, so it might not be what you want (hyperbolic structure is different than Euclidean space, which is used for most word embeddings).

Improve this answer
$\endgroup$
1
  • $\begingroup$ What I meant was concepts which cannot be easily described by one or two words and consist of multiple lower level concepts. Both node2vec and Poincare embeddings seem like nice approaches since they seem to take into account hierarchical structure of the knowledge base, which would correspond to my notion of low/high-level concepts. Thank you so much for these suggestions Jakub! $\endgroup$
    – slazien
    Commented Dec 18, 2017 at 22:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.