Questions tagged [bag-of-words]
A way of representing language data that consists of the constituent words w/ their individual frequencies. Ie, grammar & order, etc, are dropped to simplify the data.
37 questions
2
votes
2
answers
46
views
End-Tokens are Required to make Ngram Models Proper
The standard bigram model, (for example defined here) defines a probability distribution over a corpus $V$ based on the following principles:
The marginal probability of a word $w$ is defined as its ...
- 73
0
votes
1
answer
14
views
Continuous Bag of Words derivation
The continuous bag of words model has the following log probability for observing a sequence of words: $$\log P(\textbf{w})=\sum_{c=1}^{C}\log{P(w_c|w_{c-m},...w_{c-1}, w_{c+1},...,w_{c+m}})$$
I don't ...
- 319
1
vote
0
answers
25
views
What is a word embedding approach that would work for these pre-labeled documents?
My Situation:
I should start off with my end goal: I want to get a distance metric between each document and all of the other documents
To get there, I first need to encode these topic labels so that ...
- 595
0
votes
0
answers
47
views
Continuous Bag of Words NY Time Corpus
I am working to implement the continuous bag of words approach on the New York Times corpus dataset. However, I am getting word embeddings that do not seem very useful based on a few examples of ...
- 51
0
votes
0
answers
53
views
why is using a small vocabulary for topic modelling bad?
i am trying to classify texts into topics. for example, let's say one of the topics is cooperation. so in the vocab param of the sklearn api. so some of the prevalent words (or "tokens" are ...
- 101
0
votes
0
answers
84
views
Fast feature selection on a huge dataset in R on a term document matrix
I have a 500K rows x 10K features dataset.
It consists into :
a term document matrix with words + bigram and TF-IDF weighting
6 one hot encoded multi-labels
It is much more features that I want to ...
- 111
1
vote
1
answer
201
views
BOW features classifying better than complex models like BERT
I am doing a document classification task and I find that using simple BOW features with a random forest provide better results than using complex models like BERT or ELECTRA even after doing some ...
- 149
0
votes
1
answer
24
views
Transforming topics into text data
I was reading some articles on topic classification, in which some algorithm uses snippets of text as input and tries to classify them in topics, and I thought of implementing this technique in my ...
- 113
2
votes
3
answers
437
views
How can "word hashing" cause a collision in DSSM?
They say in their paper, that "word hashing" can cause a collision. But I don't understand, how. For example, if word good is tranformed to ...
- 412
1
vote
0
answers
201
views
Language Identification Better Results with Unigrams
I have a school project which consists of identifying each language of a tweet from a dataset of tweets. The dataset contains tweets in Spanish, Portuguese, English, Basque, Galician and Catalan. The ...
user255085
1
vote
2
answers
512
views
Why CBOW model is called "continuous"?
The question is pretty clear from the Title itself, why the Continuous Bag of Words (CBOW) model is called continuous.
I also don't know what exactly "distributed representation" of word vectors ...
- 155
2
votes
1
answer
908
views
could someone please give a concrete example to illustrate the Dirichlet distribution prior for bag-of-words?
I am aware of the notion of the Dirichlet distribution, a multivariate generalization of the beta distribution.
To get parameters of the Dirichlet distribution prior for bag-of-words, this CMU ...
- 2,005
3
votes
2
answers
263
views
could someone please give an concrete example to illustrate what does Multiplicity mean in the context of Bag-of-words model?
This CMU Machine Learning Course is using the Bag-of-words model without too much explanation.
wiki uses the term multiplicity to explain that model.
The bag-of-words model is a simplifying ...
- 2,005
1
vote
2
answers
735
views
Text classification with small dataset for a specialized domain
I have a multiclass text classification problem where I have very few documents for each class. The classes are imbalanced but I want to be able to predict the class when I have at least 200 - 300 ...
- 77
1
vote
0
answers
71
views
Bag of Visual Words: is feature extraction even needed?
I'm currently implementing a BoVW as part of my lab project. The steps the algorithm used are as follows:
spliting all photos into patches
cluster these pathces using K-means based on pixel values of ...
- 107
1
vote
1
answer
652
views
Classification using n-grams [closed]
I have $10000$ samples of 6-lettered strings of the following type
Left Right &...
- 299
1
vote
1
answer
22
views
Group of word representations
For word representation baseline people use bag-of-words or word embedding. Here, I want to understand all approaches that can be used for word representations. For example:
-Bag-of-words (tfidf, n-...
- 11
3
votes
1
answer
2k
views
Understanding the role of document size parameters in Latent Dirichlet Allocation
I am writing a pymc3-based implementation of Latent Dirichlet Allocation, and am referencing this CrossValidated answer (modified for pymc3) as well as pymc3's own tutorial on LDA, in addition to the ...
- 2,352
7
votes
1
answer
3k
views
Word2Vec : Difference between the two Weight matrices
In Word2Vec algorithm, two weight matrices are learnt :
W : Input-hidden layer matrix
W': Hidden-output layer matrix
For reference, CBOW model architecture:
Why is W chosen to represent the word ...
- 73
11
votes
5
answers
2k
views
How to measure dispersion in word frequency data?
How can I quantify the amount of dispersion in a vector of word counts? I'm looking for a statistic that will be high for document A, because it contains many different words that occur infrequently, ...
- 317
1
vote
0
answers
82
views
What is the difference between training examples generated by continuous bag of words (CBOW) and skip-gram?
This is a simple question that is hard for me:
Let's consider simple sentence
A B C D
and create training examples for skip-gram training (x, y) with number of ...
- 498
1
vote
1
answer
436
views
Which classifiers do consider the order of the features?
In case the order of features can make a difference in the results of a classification approach, which classifier algorithms perform better? I know Naive Bayes/KNN use bag of words and ignore the ...
- 123
3
votes
1
answer
1k
views
Regularization in text classification with bag-of-words
I am performing a text categorization with bag of words and logistic regression.
I have already heard about L1 and L2 regularization and used them for classification but with problems handling way ...
- 203
1
vote
0
answers
412
views
Low score in sentiment analysis : how to increase it and maybe deal with class imbalance
It has been 2 weeks now I am working on SemEval task 4 (2016) : Sentiment Analysis on Twitter.
The results I achieve are lower than what I expected for the three class classification problem : ...
- 203
2
votes
1
answer
690
views
Find most similar sentence from one list of sentences to another
I have two lists of short sentences (List A and List B). For each short sentence in List A, I am trying to find the most similar short sentence in List B.
Each list has a different count of elements ...
- 21
1
vote
0
answers
365
views
Creation and validation of cluster for Bag of words
I recently came across a problem where I have been given a dataset of Bag of words, the description of the dataset is given in the readme file.
What I have been trying to do is create clusters of ...
- 111
0
votes
0
answers
1k
views
Combatting data sparsity, overfitting in bag of words model
I am looking at plots of learning curves (accuracy vs training examples) in order to compare different feature extraction methods that I am trying using a bag of words model (term presence vs. term ...
- 3
1
vote
1
answer
90
views
Is machine learning a viable approach to extract license references from source code files?
I am a complete newcomer to the field of machine learning. I do have a lot of experience in computer programming, but nothing related to ML.
My question is whether or not ML would be a good approach ...
- 11
2
votes
1
answer
56
views
Searching for list of terms using Google in order to build a bag-of-words for a particular category [closed]
I am having a hard time understanding the process of building a bag-of-words. This will be a multiclass classification supervised machine learning problem wherein a webpage or a piece of text is ...
user162992
3
votes
2
answers
453
views
Language Modelling using Neural Networks
I plan to make a Language Model in Python using Neural Networks. I've read that Neural Networks need vectors as input. One common vector representation in NLP is the Bag of Words model. Given a corpus ...
- 53
4
votes
0
answers
321
views
Google gender-pay gap vs
Background:
I read this:
google schools US government about gender pay gap.
It derives from this google blog post by Eileen Naughton, VP of People Operations.
She asserts that google is somehow "...
- 9,853
6
votes
1
answer
2k
views
Understanding Word2Vec
I am trying to understand the word2vec algorithm (Mikolov et. al) but there are a few thing which I do not understand.
I get that the activation from the input layer ot the hidden layer is linear and ...
- 337
1
vote
1
answer
2k
views
How does a bag-of-words model treat words that were never seen before (not in the training data)?
What happens when a text classifier using a bag-of-words model (let's say we're using logistic regression) encounters a word that the model has not seen before- aka, words that were not in the ...
- 113
2
votes
1
answer
326
views
Why do we need Tokenzier if we have Vectorizer
In the ML learning textbook I am working through, it says, that for NLP we construct a feature vector from the Text via the Bag of Words model.
For that, we are using
...
- 705
0
votes
1
answer
197
views
Calculate predictability of events over time
I'm trying to create a model / algorithm which learns the predictability of events over time, which takes into account both frequency and rarity.
An example of what this could apply to (which is the ...
- 101
3
votes
2
answers
922
views
How to asses the optimal bag of words vector size?
I have a corpus with 6040592 words and 309074 types (different words). Knowing this information it is possible to know the optimal size of bag of words vectors in order to represent phrases?
I am ...
- 141
1
vote
0
answers
61
views
Treating numerals/cardinals in Bag of Words (BOW) model
I wish to do topic modeling on text corpus some of which are about company earnings which has lots of numbers in it. It has no sentence structure. I think tagging numbers using nltk.pos_tagging can ...
- 343