this NLP book gives

When we use a bigram model to predict the conditional probability of the next word, we are thus making the following approximation:

$P(w_n|w_1^{n-1}) \approx P(w_n|w_{n-1})$

does $w_1^{n-1}$ denote whole the sentence?


On page 3, the notation is defined:

We'll represent a sequence of $N$ words either as $w_1 \ldots w_n$ or $w_1^n$ (so the expression $w_1^{n-1}$ means the string $w_1, w_2, \ldots, w_{n−1}$).

So yes, it represents the sequence of words from $w_1$ to $w_{n-1}$, i.e. the whole sentence up to that point.

  • $\begingroup$ However, in the special case where the sequence $w_i$ is a Markov chain, then knowing $w_{n-1}$ is the same as knowing $w_1^{n-1}.$ $\endgroup$ – BruceET Jul 26 '19 at 3:48
  • $\begingroup$ Thanks for you answer. $w_1^n$. Here has nothing to do with exponential computation, right? $\endgroup$ – czlsws Jul 27 '19 at 22:49
  • $\begingroup$ Nothing at all, they're just using the superscript to differentiate the two symbols. $\endgroup$ – ConMan Jul 29 '19 at 0:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.