How does two tower model map to shared embedding space for two different type of entity?

Question

A canonical example is say you have user and merchandise

user (feature: age, location....) merchandise (feature: type, size, .....)

And you want to create embedding to map user and merchandise to same space

My understanding of two tower model is that you can train two DNN encoder

DNN_encoder_for_item DNN_encoder_for_user

together

using input(either user or merchandise with label) and neural network to train the model

Once you have the model you can use the model to generate embeddings using the output of last few layers of neural network

What I am confuse is the following

1. since user and merchandise is different entity, how do you label two different entities such that they are same context?

2. is both DNN encoder derive from the same model? or do we train two models?

3. how does the embedding map different entity to the same space?

Answer 1

The key idea behind two tower model in recommendation systems is to map both entities to the same embedding space where similar entities are close to each other.

Through deep neural networks, the model is able to learn high-level abstract representations for both a user and an item with past user-item interactions. The output is the similarity between user embedding and item embedding, which represents how interested the user is in the given item.

The label in this context is typically derived from user interactions with merchandise, such as a user rating an item or clicking on an item. These interactions (user, interacted item) serve as positive signals that the user and the item should be close together in the embedding space. Conversely, random pairs (user, unrelated item) can be used as negative labeled samples to push the embeddings of these pairs further apart.

The two DNN encoders are not the same model but they are trained together in a shared framework so that their embeddings are comparable in the same space. Each encoder is specifically designed to handle the distinct features of users and items. And some distance metric is usually used to represent the similarity as the output of the two tower model.

To ensure sharing the same embedding space, firstly each encoder network ensures to create same dimensional embedding layer from their respective feature input layer. Secondly the two encoders must be jointly learned based on minimizing the meaningful contrastive or triplet loss dependent on the above similarity metrics of all paired positive/negative contrastive training data, similar to the early Siamese network used for metric learning.

Learning in twin networks can be done with triplet loss or contrastive loss... The common learning goal is to minimize a distance metric for similar objects and maximize for distinct ones. This gives a loss function like $${\displaystyle {\begin{aligned}\delta (x^{(i)},x^{(j)})={\begin{cases}\min \ \|\operatorname {f} \left(x^{(i)}\right)-\operatorname {f} \left(x^{(j)}\right)\|\,,i=j\\\max \ \|\operatorname {f} \left(x^{(i)}\right)-\operatorname {f} \left(x^{(j)}\right)\|\,,i\neq j\end{cases}}\end{aligned}}}$$ ${\displaystyle i,j}$ are indexes into a set of vectors, ${\displaystyle \operatorname {f} (\cdot )}$ function implemented by the twin network.