I am new to NLP and Neural Networks. I want to do topic analysis for a dataset of reviews of our product. I tried to use the universal-sentence-encoder along with top2vec and they do a good job. However, the similarity score is low most of the time because the combination of words is unique. I want to retrain the model to capture the similarity in my dataset (which is about 40k reviews).

Is there a way to do so? I think this is called unsupervised fine-tuning

I am aware of Keras' API to import the transformer as a layer, but I do not how to continue from there.

Question 1: What layers or loses could I add to capture similarity between my reviews and outputs vector embedding of the text?

I also read about siamese networks. If I understood correctly, I can add the universal encoder twice as the common network, then add a layer for similarity like keras.layers.Dot.

Question 2: If I trained the model on every possible combination of me reviews, can I then use the output of the first embedding layers (with the new weights) as the embedding of text?

Question 3: If the answer for question is yes, both embeddings in the siamese networks should be almost identical for the same text. Right?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.