# Why K and V are not the same in Transformer attention?

My understanding is for translation task K should be the same with V, but in Transformer K and V are generated by two different(randomly initialized) matrix $$W^K, W^V$$, therefore not the same. Can any one tell me why?

• why do you think k should be the same as v? – shimao Oct 14 '19 at 2:46
• We compute similarity of Q and K and use its result as weight to pass K information into the new value of Q, V and K should be the same so that Q know how much K's information should be added to Q – eric2323223 Oct 14 '19 at 6:09
• does this answer help? stats.stackexchange.com/a/424127/95569 – dontloo Oct 16 '19 at 2:45