Timeline for Curse of dimensionality with language models
Current License: CC BY-SA 3.0
6 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Dec 2, 2015 at 8:41 | vote | accept | Antoine | ||
| Dec 1, 2015 at 15:55 | comment | added | kjetil b halvorsen♦ | subtracting one does not by itself secure that the probabilities will sum to one, but it takes care of that restriction . After freely choosing the $100000^{10}-1$ probabilities, you can calculate the last one! | |
| Dec 1, 2015 at 15:34 | comment | added | Antoine | Many thanks for elaborating. In other words, $100,000^{10}$ gives all the possible 10-element combinations from the vocabulary of size $100,000$. This number is huge but still finite since we're in the discrete case. And estimating the joint probability mass function of a specific 10-word sequence comes down to assigning a probability (probabilities=parameters here I assume) to each portion of that huge but finite discrete sample space. Is that correct? Also, sorry if I'm being dumb but I still don't get why subtracting 1 ensures that the probabilities will sum to one. | |
| Dec 1, 2015 at 14:43 | history | edited | kjetil b halvorsen♦ | CC BY-SA 3.0 |
added 746 characters in body
|
| Dec 1, 2015 at 14:30 | comment | added | Antoine | please elaborate. Is the fact that words are consecutive important? How does joint distributions and degrees of freedom come into play? What is the formula that is used? | |
| Dec 1, 2015 at 14:25 | history | answered | kjetil b halvorsen♦ | CC BY-SA 3.0 |