All Questions
6 questions
1
vote
1
answer
86
views
Do recurrent neural language models greedily model language probability?
Want to check my understanding of recurrent neural language models (in this case I'm working with a decoder in an encoder-decoder RNN but I don't think that matters significantly). I'm trying to ...
- 13
1
vote
0
answers
35
views
Character level RNN for converting word forms
I want to build a char RNN to convert word form from one to another, for example, singular nouns such as lion to lions. However ...
3
votes
1
answer
220
views
How do we pass data to a RNN?
Let's say we have A1, A2, ... , Am different articles in the corpus and each of them has W1, W2, ....., Ww words. We are training a language model on them. Do we:
Scheme 1
Take the first batch of ...
- 2,693
2
votes
0
answers
451
views
Deciding on the training sequences for RNN/LSTM language model
In a character language model, text is seen as a stream of characters. Say we have a training text as a string s, with length ...
- 692
7
votes
2
answers
3k
views
Calculating test-time perplexity for seq2seq (RNN) language models
To compute the perplexity of a language model (LM) on a test sentence $s=w_1,\dots,w_n$ we need to compute all next-word predictions $P(w_1), P(w_2|w_1),\dots,P(w_n|w_1,\dots,w_{n-1})$.
My question ...
- 96
5
votes
1
answer
8k
views
Advantage of character based language models over word based
Is there an intuition why character based models language bases models are preferred over word based. For example Karpathy builds his language model by predicting the next character in Karpathy Blog.
...
- 211