1
$\begingroup$

I do not understand the following part of this paper (page 4):

enter image description here

Can someone explain how this math gives us soft alignments between similar words?

Thanks

$\endgroup$

1 Answer 1

1
$\begingroup$

I believe I may be comepletley missing the point, since I am relativley new to the subject but i think that the math doesnt 'give' us soft alignments between similar words, but more so enables the possibility of it.

I assume that the neural network "Learns" to make a soft alignment out of this particular construction since semnatic resemblence can in fact be derived from word embeddings, and moreover the ability to "focus", provide more "attention" on words of similar meaning in the question and passage truly does help the process of text comprehension.

the math "enables" soft alignment since it provides a way to use information from word embeddings (since the word embeddings are used in the equation) pass them through a neural network (i.e let the neural network learn the way to maximize dot product of similar semantic values) and translate that resemblence to a probability distrubtion (using the softmax to "rank" the similarities of passage words to question words).

Thus the math provides the tools to perform the soft alignment, and the soft alignment is performed since it has turned out to be valuable for reading comprehension, the training found it through it being the best way available through the existing neural architecture

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.