How does tf.keras.MultiHeadAttention layer handle positional encoding?

Question

In Attention Is All You Need paper, positional encodings are added to the input embeddings in order to consider the order of the sequence. How does tf.keras.MultiHeadAttention handle positional encoding, given that it is an implementation of such paper? Could I use my own custom positional encodings by adding them to the input sequence, right before using the tf.keras.MultiHeadAttention layer?

Documentation for this layer doesn't say anything about positional encoding.

Answer 1

It doesn't. Positional encoding is not part of the multi-head attention layer.