All Questions
Tagged with language-models distributions
5 questions
2
votes
2
answers
46
views
End-Tokens are Required to make Ngram Models Proper
The standard bigram model, (for example defined here) defines a probability distribution over a corpus $V$ based on the following principles:
The marginal probability of a word $w$ is defined as its ...
- 73
0
votes
0
answers
98
views
Normalize probability distribution by variance of each class
I have several topics that I will collect language data for. Using Mturk I will ask responders to write sentences for each topic. The sentences will be used to train a language model. Language models ...
- 51
6
votes
1
answer
525
views
n-gram language model
At the end of the introduction of A Neural Probabilistic Language Model (Bengio et al. 2003), the following example is given:
Having seen the sentence ...
- 6,217
1
vote
1
answer
511
views
Curse of dimensionality with language models
In the seminal paper A Neural Probabilistic Language Model, Yoshua Bengio and his colleagues make the following point:
If one wants to model the joint probability distribution of 10
consecutive ...
- 6,217
8
votes
1
answer
268
views
Language modeling: why is adding up to 1 so important?
In many natural language processing applications such as spelling correction, machine translation and speech recognition, we use language models. Language models are created usually by counting how ...
- 183