I am searching for the intuitive difference between Bag-of-words and vector space model? Is there any relationship exists between bag-of-words and vector space model. I tried searching but couldn't find satisfactory answer.
Bag of words vs vector space model? has also been asked but not answered in a satifactory way.
$\begingroup$
$\endgroup$
0
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
I find existing answer very misleading.
The Word vector (aka word embedding) is concept coming from probabilistic language models (see [1]). It describes contextual similarity between the words in the language model and came into existence several decades after the VSM was proposed and successfully applied for text categorization, document summarization and information retrieval.
In the Vector Space Model (see [2]), it is not word/term being represented as vector in a n-dimensional space but document. The VSM is constructed to have separate dimension for each distinct unigram word/term, existing in collection of terms aggregated from all BOWs in the document collection. In other words, in the VSM: distinct terms became dimensions, not word vectors. Documents are vectors in the VSM, located at associated term weights by each corresponding dimension.
Bag-of-words (BOW), as approach of document representation in IR, does not allow multiple instances of same word - but represents an unordered list of distinct words, associated with their frequencies in the document (see [3]).
[1] Y. Bengio, R. Ducharme, P. Vincent, C. Janvin, A Neural Probabilistic Language Model, J. Mach. Learn. Res. 3 (2003) 1137–1155. doi:10.1162/153244303322533223.
[2] G. Salton, A. Wong, C. Yang S., A vector space model for automatic indexing, Commun. ACM. 18 (1975) 613–620. doi:10.1145/361219.361220.
[3] G. SALTON, C.S. YANG, ON THE SPECIFICATION OF TERM VALUES IN AUTOMATIC INDEXING, J. Doc. 29 (1973) 351–372. doi:10.1108/eb026562.
$\begingroup$
$\endgroup$
Note that the word "bag" means multiset, i.e., it allows multiple instances for each word. Thus:
- Bag of words indicates the count of each word in the document. This simple model is used, for example, in naive Bayes
- Word vector generalizes the idea of bag of words assigning a ranking to each word in the document. Often the occurrence count, but it can also be another ranking, such as the TF-IDF
Note that each row in a Document Term Matrix (DTM) corresponds to a word vector.
