All Questions
Tagged with language-models lstm
6 questions
2
votes
1
answer
220
views
multiple likely ys for one instance of x: word prediction with LSTM
I have a ML project that is about predicting (suggesting) the next word based on the last n words, using LSTM. The output is a softmax dense layer the size of the vocabulary that shows the probability ...
- 51
0
votes
1
answer
86
views
get predictability of word given sentence in python
I have a paragraph and I want to get the probability (p(word | context) ) of each word, given previous words, for various models (e.g. pre-trained LSTM).
Where can pretrained models would allow me to ...
- 57
1
vote
1
answer
86
views
Do recurrent neural language models greedily model language probability?
Want to check my understanding of recurrent neural language models (in this case I'm working with a decoder in an encoder-decoder RNN but I don't think that matters significantly). I'm trying to ...
- 13
0
votes
1
answer
220
views
How to train a RNN language model?
I want to train a RNN-based language model from https://arxiv.org/pdf/1409.2329.pdf for next word prediction.
How to split the sentences from the dataset into input and ground truth during the ...
- 1
2
votes
1
answer
730
views
How to sample a language model?
I've successfully trained a language model using LSTMs. But I have a confusion about sampling.
On sampling, we generate a probability distribution at each time step. It will be of length vocabulary ...
- 123
3
votes
1
answer
4k
views
How to handle big vocabulary size with keras tokenizer?
I am actually working on a neural language model developed with keras.
I have an encoder and a decoder and the output of the decoder is a dense vector on the vocabulary..so quite big depending on the ...
- 31