4
$\begingroup$

In NLP, what is the difference between corpus and vocabulary?

I see these words often referred to and I feel like they are referring to the same thing. Is there a difference between them or are they the same thing?

$\endgroup$

2 Answers 2

7
$\begingroup$

They are not the same thing.

  • Corpus: Collection of texts used to train an NLP model.
  • Vocabulary: Collection of words used to train an NLP model.

It might be easier to explain by example: BERT is an advanced NLP model trained on the entire content of Wikipedia (originally the English language Wikipedia). The corpus is the collection of Wikipedia articles it was trained on. The vocabulary is the vocabulary of the English language.

$\endgroup$
2
  • $\begingroup$ Thanks for the clarification. In your example, could each Wikipedia article in the corpus also be referred to as a 'document'? $\endgroup$
    – zipline86
    Jun 17, 2020 at 11:18
  • 1
    $\begingroup$ @zipline86 in theory yes. In practice, if you use the term "document" you might mislead people, because the term "document" has a very specific meaning in the context of some NLP models, such as Doc2Vec. $\endgroup$
    – Skander H.
    Jun 17, 2020 at 21:18
0
$\begingroup$

A vocabulary is just a collection of words. Mathematically, members of a vocabulary are members of a set because each member is a unique instance. There is no order or structure between words in a vocabulary.

By contrast, words in a corpus are not members of a set.

As a @Skander described, a corpus is a collection of text. This text reflects the usage of the words in a vocabulary. A corpus has structure and the meaning (semantics) of words within a corpus rely heavily on this structure (context) to derive meaning.

$\endgroup$
2
  • $\begingroup$ so when a corpus says it has billions of words, it's all of the words in the text collection without deduplication yes? $\endgroup$
    – cifer
    Mar 16, 2022 at 8:42
  • 1
    $\begingroup$ Yes. That's how I'd read it. $\endgroup$ Mar 16, 2022 at 15:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.