All Questions
Tagged with language-models probability
5 questions
2
votes
2
answers
46
views
End-Tokens are Required to make Ngram Models Proper
The standard bigram model, (for example defined here) defines a probability distribution over a corpus $V$ based on the following principles:
The marginal probability of a word $w$ is defined as its ...
- 73
2
votes
1
answer
220
views
multiple likely ys for one instance of x: word prediction with LSTM
I have a ML project that is about predicting (suggesting) the next word based on the last n words, using LSTM. The output is a softmax dense layer the size of the vocabulary that shows the probability ...
- 51
4
votes
1
answer
3k
views
Language Model compare probability scores between Length varying sentence
My question is : How can I compare Language Model(LM) score for two sentences with different lengths ?
Probabilities are < 1...
5
votes
2
answers
12k
views
Understanding Add-1/Laplace smoothing with bigrams
I am working through an example of Add-1 smoothing in the context of NLP
Say that there is the following corpus (start and end tokens included)
...
- 173
6
votes
1
answer
525
views
n-gram language model
At the end of the introduction of A Neural Probabilistic Language Model (Bengio et al. 2003), the following example is given:
Having seen the sentence ...
- 6,217