So one of my projects is to build a bot that brings forth relevant pieces of a document based on an input doc.

For example, if my reference document is the bible:

input doc: 'A mother gave birth to a child, in a barn where she clothed him. The inn was full"

returned doc: 'and she gave birth to her firstborn, a son. She wrapped him in cloths and placed him in a manger, because there was no guest room available for them.'

I already have a decent enough model just by converting my reference doc and input docs into word vectors with spacy, and then performing a cosine similarity search. The result brings up relevant documents, but almost never brings forth the specific line that I am paraphrasing.

I would like to sit down and paraphrase a bunch of lines, match them with their source in the reference doc, and then use those pairings to train a neural network, but I'm stuck on the type of network to use.

the input would be two documents, and the output would be a similarity vector between (0, 1). I've debated using a simple CNN or maybe even an attention mechanism as my model, but I don't see how I would structure the network. Especially when it comes to representing the textual documents as numbers (do I use word vectors? Sparse vectors of counts? Dense vectors with word IDs?).

Do you have any insights into the best models for custom document matching? Or perhaps a better data preprocessing technique?

P.S. I am using Keras and Spacy in Python

  • $\begingroup$ Maybe this repo would be of any help? github.com/life4/textdistance If you need semantic similarity it's a training task and the key problem is training corpus. $\endgroup$ Feb 18, 2020 at 0:18
  • $\begingroup$ I need more of abstract contextual similarities. The kind that word vectors produce. I dont think semantic similarities will get me there. For example: 'Barn' and 'manger' need to be rated as a high similarity. $\endgroup$ Feb 18, 2020 at 0:39


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.