4
$\begingroup$

The skip-gram model of word2vec uses a shallow neural network to learn the word embedding with (input-word, context-word) data. When I read the tutorials for the skip-gram model there was not any mentioning regarding the N-gram. However I came across several online discussions in which people claim --- skip-gram model in word2vec is an expanded version of N-Gram model. Also I don't really understand this "k-skip-n-gram" in the following Wikipedia page.

Wikipedia cited a paper from 1992 for "skip-grams", so I guess this is not the word2vec's skip-gram model, right? Another paper regarding this "skip-grams" is https://homepages.inf.ed.ac.uk/ballison/pdf/lrec_skipgrams.pdf. This is very confusing. Why there's no one clear this up.

The wikipedia source and the online discussion are as follows:

$\endgroup$

1 Answer 1

2
$\begingroup$

Some basic concepts are valid through years :) and are used in many solutions and naturally contributing to naming of these solutions...

N-gram is a basic concept of a (sub)sequnece of consecutive words taken out of a given sequence (e.g. sentence).

k-skip-n-gram is a generalization where 'consecutive' is dropped. It is 'just' subsequence of the original sequence, e.g. every other word of the sentence is 2-skip-n-gram.

word2vec is more complicated beast, the buzzword :) here is 'embeddings', here is the original paper https://arxiv.org/pdf/1301.3781.pdf. It uses the concept of consequtive words and 'skip' and 'gram' made its way to the name of the algorithm. BTW there are two alternative ones used by word2vec solution: skip-gram and CBOW.

$\endgroup$
2
  • $\begingroup$ So there is no direct relation between k-skip-n-gram and skip-gram? $\endgroup$ Aug 31, 2018 at 5:41
  • 1
    $\begingroup$ k-skip-n-gram (or shortly skip-gram in this context) is a general concept of 'skipping some words in a sequence (e.g. sentence) while in context of word2vec it is the name of one of its algorithms - much more complex than 'just' the idea of skipping some words but it uses a context with a 'skipped word'. $\endgroup$
    – MkL
    Sep 1, 2018 at 7:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.