When reviewing Infersent's architecture here, I noticed that, after encoding the premise and hypothesis to obtain two vectors u and v, they feed the set of fully connected layers with:
- (u, v) the concatenation between u and v,
- u * v the element-wise product,
- |u - v| the absolute element-wise difference
While I can somewhat get a feel for why points 1 and 3 were used, I do not really understand how an element-wise product can help in this case.
My intuition for 1 and 3 is the following:
- Point 1 was used because it makes sense to feed the actual encodings of the two sentences
- The element-wise difference gives a sense of the similarity between the two sentence encodings
Does anyone know, why the element-wise product would help?
PS: A picture of the infersent architecture can be found below (extracted from the paper).