1
$\begingroup$

I am new to NLP and Neural Networks. I want to do topic analysis for a dataset of reviews of our product. I tried to use the universal-sentence-encoder along with top2vec and they do a good job. However, the similarity score is low most of the time because the combination of words is unique. I want to retrain the model to capture the similarity in my dataset (which is about 40k reviews).

Is there a way to do so? I think this is called unsupervised fine-tuning

I am aware of Keras' API to import the transformer as a layer, but I do not how to continue from there.

Question 1: What layers or loses could I add to capture similarity between my reviews and outputs vector embedding of the text?

I also read about siamese networks. If I understood correctly, I can add the universal encoder twice as the common network, then add a layer for similarity like keras.layers.Dot.

Question 2: If I trained the model on every possible combination of me reviews, can I then use the output of the first embedding layers (with the new weights) as the embedding of text?

Question 3: If the answer for question is yes, both embeddings in the siamese networks should be almost identical for the same text. Right?

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.