Questions tagged [language-models]
A statistical language model is a probability distribution over sequences of words.
4 questions
11
votes
3
answers
16k
views
Regarding using bigram (N-gram) model to build feature vector for text document
A traditional approach of feature construction for text mining is bag-of-words approach, and can be enhanced using tf-idf for setting up the feature vector characterizing a given text document. At ...
- 3,089
3
votes
2
answers
2k
views
Language Detection with CLD2 with Mixed Inputs in long documents
Internals Recap. CLD2 is a Naïve Bayesian classifier, trained on documents of mean size of 200 characters, trained on a corpus of 100M scraped and human expert selected web pages.
When working on ...
- 153
2
votes
1
answer
220
views
multiple likely ys for one instance of x: word prediction with LSTM
I have a ML project that is about predicting (suggesting) the next word based on the last n words, using LSTM. The output is a softmax dense layer the size of the vocabulary that shows the probability ...
- 51
1
vote
1
answer
511
views
Curse of dimensionality with language models
In the seminal paper A Neural Probabilistic Language Model, Yoshua Bengio and his colleagues make the following point:
If one wants to model the joint probability distribution of 10
consecutive ...
- 6,217