1
$\begingroup$

I found a nice tutorial here regarding CBOW and Skip-gram models for Word2Vec. I got the following doubt:

Why two layers of matrix multiplication is need for CBOW and SKIP-GRAM ? As multiplying a vector with a Matrix followed by another matrix can be written as multiplication by an equivalent single matrix.

$\endgroup$
2
$\begingroup$

Multiplying a vector with a matrix A followed by another matrix B can be written as multiplication by an equivalent single matrix. However, the number of elements in the matrix AB can be much higher than number of elements in the matrix A + number of elements in the matrix B.


Example in CBOW:

enter image description here

$A = W$, $B = W'$. The number of elements in the matrix AB is $N^2$, while the number of elements in the matrix A + number of elements in the matrix B is $2VN$. (and $V \gg N$). Hence the use of two distinct matrices.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.