0
$\begingroup$

I've tried out my code just to make sure it works on a dataset of 100 input and output sentences. Turns out that no matter what I insert, the trained model only responds with either 1 of 2 phrases.

I convert the sentences into one-hot vectors, and my matrix is very sparce, in other words, I have some very long sentences and some very short ones, and the short ones are padded with zero vectors. I also do not implement attention and beam search.

Is this because of the incredibly small dataset, or does that normally not occur?

$\endgroup$
1
  • $\begingroup$ It's impossible to answer this without seeing the data and running reproducible code. $\endgroup$
    – Alex R.
    Jan 9 '18 at 20:20
1
$\begingroup$

Sounds like small and biased dataset to me, which assigns high biases to those 2 neurons on output layer. Take a look at the biases in the last layer.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.