# How does Transformer use BPE?

I'm trying to re-implement Transformer on my own but I'm struggling on the input encoding.

It seems that the original paper(Attention is All You Need) mentioned that they use Byte-pair Encoding to encode words into vectors but as I understand BPE is just used for word segmentation (compress size of the dictionary) so how will it produce a numeric vector?

In many implementations, I saw that they just initialled random weights for word embedding. In this case, will the weights be updated when the model is being trained?

Lastly, As they mentioned that "we share the same weight matrix between the two embedding layers and the pre-softmax linear transformation", I understand that two embedding layers could be shared but why "the pre-softmax linear transformation" should be the same matrix to the embedding? It doesn't make sense to me

In this paper: Using the Output Embedding to Improve Language Models the author found that sharing the parameters would benefit the performance. The author also found that in Word2vec the $$W'$$, which is also an embedding matrix, is kind of inferior to the standard embedding matrix $$W$$. Some guys tried to tie these two matrices but that leads to the detriment. But in large neural network language models, such parameter sharing is reportedly benefitial.