8
$\begingroup$

In word2vec's CBOW and skip-gram models, how does choosing word vectors from $W$ (input word matrix) vs. choosing word vectors from $W'$ (output word matrix) impact the quality of the resulting word vectors?

CBOW:

enter image description here

Skip-gram:

enter image description here

$\endgroup$
7
$\begingroup$

Garten et al. (1) compared word vectors obtained by adding input word vectors with output word vectors, vs. word vectors obtained by concatenating input word vectors with output word vectors. In their experiments, concatenating yield significantly better results:

enter image description here


(1) Garten, J., Sagae, K., Ustun, V., & Dehghani, M. (2015, June). Combining Distributed Vector Representations for Words. In Proceedings of NAACL-HLT (pp. 95-101).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.