Due to architecture choices and organization of code, I have a file called data.py that processes texts and returns two vectors : X and Y which are the vectorized text and the corresponding label. Then I have a model.py file that takes X and Y as an input and trains on this data.
In fact, the file data.py contains different processing functions that process the text differently (word2vec, doc2vec, tfidf etc.). Unfortunately I have one processing function that must also return an embedding matrix so that the model can create an embedding layer. This breaks the pattern of only returning X and Y for all the processing functions.

To hack this, my idea was to embed the vectors directly into data.py and return the corresponding X (and Y). As a result, the model will not need an embedding layer which is not a problem, except if I want to change the weights of this embedding as the model is training. My question is then the following : is it possible to bypass this problem by adding a fully connected layer as the first layer of the model ? This FC would take as an input the embedded vectors and would be able to 'simulate' the modification of the weights of the embedding layer as it was done when the model had one. Does it make sense ?

Thank you for your help and insights.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.