0
$\begingroup$

I am working on a project to read up a text segment and find answers to a specific set of questions, in order to do some information extraction.

I have a set of text corpus (each of about 3000 words), and I have about 10 questions to get the important facts out of the text. However, I do not have a set of answers to these question, so I cannot take a supervised approach to train a model.

What kind of an unsupervised approach can i use to get the answers ( this can be extractive text segments) from the text corpus.

I have thought about using word embeddings to find words in the text that match, or are close to the words in the question, but I'm not too sure on how to go about this.

Any ideas would be greatly appreciated!

TIA

$\endgroup$

1 Answer 1

0
$\begingroup$

One approach could be segmenting each text into paragraphs (if you have some formatting clues like double end-of-line) or sentences. Then for each chunk (paragraph / sentence), you can compare if it is similar with your questions. For similarity, you need to vectorize both question and chunk and here word embeddings can help although you need to come up with vectors for these chunks / questions, so more like doc2vec approach or manually averaging word vectors. In the latter case, I would suggest weighing with tf-idf of words.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.