# First, understand ```Q``` and ```K```
First, focus on **the objective of ```First MatMul```** in the [Scaled dot product attention][1] using ```Q``` and ```K```.
[![enter image description here][2]][2]
## Intuition on what is ```Attention```
For the sentence **"jane** visits africa".
When your eyes see ***jane***, your brain looks for **the most related word** in the rest of the sentence to understand what **jane** is about (query). Your brain focuses or attends to the word **visit** (key).
This process happens for each word in the sentence as your eyes progress through the sentence.
## First MatMul as Inquiry System using Vector Similarity
The first ```MatMul``` implements an inquiry system or question-answer system that imitates this brain function, using Vector Similarity Calculation. Watch [CS480/680 Lecture 19: Attention and Transformer Networks][2] by professor Pascal Poupart to understand further.
> Think about the attention essentially being some form of approximation of SELECT that you would do in the database.<br>
> [![enter image description here][3]][3]
[![enter image description here][4]][4]
Think of the MatMul as an inquiry system processes the inquiry: "For the word **```q```** that your eyes see in the given sentence, what is the most related word **```k```** in the sentence to understand what **```q```** is about?"
The inquiry system provides the answer as the probability.
| q | k | probability |
| ------------- |-------------| -----|
| jane | visit | 0.94 |
| visit | africa| 0.86 |
| africa | visit | 0.76|
Note that the softmax to normalize the probability (in green) has been implicitly incorporated in the diagram.<br>
[![enter image description here][5]][5]
There are multiple ways to calculate the similarity between vectors such as cosine similarity. Transformer attention uses simple **dot product**.
## Where are ```Q``` and ```K``` are from
The transformer encoder training builds the weight parameter matrices ```WQ``` and ```Wk``` in the way ```Q``` and ```K``` builds the Inquiry System that answers the inquiry "**What is ```k``` for the word ```q```**".
The calculation goes like below:
1. Picks up a word vector (position encoded) from the input sentence sequence, and transfer it to a vector space **Q**. This becomes the **q**uery.<br>
$Q = X \cdot W_{Q}^T$
2. Pick all the words in the sentence and transfer them to the vector space **K**. They become keys and each of them is used as **k**ey.
$K = X \cdot W_K^T$
3. For each (**q**, **k**) pair, their relation strength is calculated using dot product.
$q\_to\_k\_similarity\_scores = matmul(Q, K^T)$
4. Weight matrices $W_Q$ and $W_K$ are trained via the back propagations during the Transformer training.
We first needs to understand this part that involves **Q** and **K** before moving to ***V***.
[![enter image description here][6]][6]
# Then, understand how ```V``` is created using ```Q``` and ```K```
## Second Matmul
Self Attention then generates the embedding vector called **attention value** as a bag of words where each word contributes proportionally according to its relationship strength to **q**. This occurs for each **q** from the sentence sequence. The embedding vector is encoding the relations from **q** to all the words in the sentence.
[![enter image description here][7]][7]
# References
There are multiple concepts that will help understand how the self attention in transformer works, e.g. embedding to group similars in a vector space, data retrieval to answer query Q using the neural network and vector similarity.
* [Transformers Explained Visually (Part 2): How it works, step-by-step][9] give in-detail explanation of what the Transformer is doing.
* [CS480/680 Lecture 19: Attention and Transformer Networks][10] - This is probably the best explanation I found that actually explains the attention mechanism from the database perspective.
* [Illustrated Guide to Transformers Neural Network: A step by step explanation][11] <br>[![enter image description here][12]][12]
* [Distributed Representations of Words and Phrases and their Compositionality][13] - It helps understand how word2vec works to group/categorize words in a vector space by pulling similar words together, and pushing away non-similar words using negative sampling.
* [Generalized End-to-End Loss for Speaker Verification][14] - Continuation to understand embedding to pull together siimilars and pushing away non-similars in a vector space.
* [Transformer model for language understanding][15] - TensorFlow implementation of transformer
* [The Annotated Transformer][16] - PyTorch implementation of Transformer
---
# Update
[Getting meaning from text: self-attention step-by-step video][17] has visual representation of query, key, value.
[![enter image description here][18]][18]
[1]: https://www.tensorflow.org/text/tutorials/transformer#scaled_dot_product_attention
[2]: https://i.sstatic.net/MJIyF.png
[3]: https://i.sstatic.net/nVvt9m.png
[4]: https://i.sstatic.net/kOUVLm.png
[5]: https://i.sstatic.net/rQhuQ.jpg
[6]: https://i.sstatic.net/DWNTr.jpg
[7]: https://i.sstatic.net/TBpsF.png
[8]: https://towardsdatascience.com/transformers-explained-visually-part-1-overview-of-functionality-95a6dd460452
[9]: https://towardsdatascience.com/transformers-explained-visually-part-2-how-it-works-step-by-step-b49fa4a64f34
[10]: https://www.youtube.com/watch?v=OyFJWRnt_AY
[11]: https://www.youtube.com/watch?v=4Bdc55j80l8
[12]: https://i.sstatic.net/xALqg.png
[13]: https://arxiv.org/abs/1310.4546
[14]: https://arxiv.org/abs/1710.10467
[15]: https://www.tensorflow.org/text/tutorials/transformer
[16]: http://nlp.seas.harvard.edu/2018/04/03/attention.html
[17]: https://peltarion.com/blog/data-science/self-attention-video
[18]: https://i.sstatic.net/ksCex.png