I use 2 layer CNN network for NLP task with triplet loss with margin 0.2. The task is to learn document embeddings to find similar docs. My architecture is similar to this https://arxiv.org/abs/1406.3830
I use truncated_normal init function from TensorFlow to initialize conv weights. I also use AdamOptimizer with default params. Then I subsample small (or big) dataset and use 5-10 epochs to train on it. But the loss stays close to 0.2 all the time. I am defiantly underfitting. But the underfitting is not related to insufficient number of layers because same architecture works fine in literature. The problem I guess is the initialization function of convolution layers weights - it is just stuck in some local minimum. But I don't know for sure.
Where is my mistake? What is a good way to init my conv layers?