What exactly are "input" and "output" word representations? So, I was reading Distributed Representations of Words and Phrases and their Compositionality, and I can't understand this part on page 3:

What exactly are these representations? One-hot vectors? Resulting embeddings? Implementation-dependent stuff? This short piece is the only time they are mentioned in the paper (at least somewhat explicitly, that is).
I just can't wrap my head around the architecture of this thing, as in "here's how it all looks in matrices and vectors form".
 A: You can look at Figure 1 in this paper: http://www-personal.umich.edu/~ronxin/pdf/w2vexp.pdf

The first matrix $W$ maps one-hot vectors to word embeddings, i.e. its rows are the "input" representations, aka the word embeddings the model produces.
The second matrix $W'$ is just a matrix of model's weights, it maps the hidden layer to the output vector of vocabulary size, to which you will apply the softmax. In the paper they call the columns of this matrix $W'$ as "output" representations.
A: There are two matrices, $V$ and $V'$. Both have their first dimension equal to the size of the vocabulary. The vector $v_w$ is the row from $V$ that corresponds to the word from the input. The vector $v'_w$ is a row from $V'$ that corresponds to the output word. 
When the word "representations" is used in this context it refers to a low-dimensional vector learned as part of the neural network. These learned representations tend to be really useful for all sorts of stuff as opposed to a one-hot encoding representation that is not useful for anything.
