Skip to main content

Questions tagged [bag-of-words]

A way of representing language data that consists of the constituent words w/ their individual frequencies. Ie, grammar & order, etc, are dropped to simplify the data.

Filter by
Sorted by
Tagged with
2 votes
2 answers
46 views

End-Tokens are Required to make Ngram Models Proper

The standard bigram model, (for example defined here) defines a probability distribution over a corpus $V$ based on the following principles: The marginal probability of a word $w$ is defined as its ...
olives's user avatar
  • 73
0 votes
1 answer
14 views

Continuous Bag of Words derivation

The continuous bag of words model has the following log probability for observing a sequence of words: $$\log P(\textbf{w})=\sum_{c=1}^{C}\log{P(w_c|w_{c-m},...w_{c-1}, w_{c+1},...,w_{c+m}})$$ I don't ...
Victor M's user avatar
  • 319
1 vote
0 answers
25 views

What is a word embedding approach that would work for these pre-labeled documents?

My Situation: I should start off with my end goal: I want to get a distance metric between each document and all of the other documents To get there, I first need to encode these topic labels so that ...
Jacob Myer's user avatar
0 votes
0 answers
47 views

Continuous Bag of Words NY Time Corpus

I am working to implement the continuous bag of words approach on the New York Times corpus dataset. However, I am getting word embeddings that do not seem very useful based on a few examples of ...
dzheng1887's user avatar
0 votes
0 answers
53 views

why is using a small vocabulary for topic modelling bad?

i am trying to classify texts into topics. for example, let's say one of the topics is cooperation. so in the vocab param of the sklearn api. so some of the prevalent words (or "tokens" are ...
yishairasowsky's user avatar
0 votes
0 answers
84 views

Fast feature selection on a huge dataset in R on a term document matrix

I have a 500K rows x 10K features dataset. It consists into : a term document matrix with words + bigram and TF-IDF weighting 6 one hot encoded multi-labels It is much more features that I want to ...
Xiiryo's user avatar
  • 111
1 vote
1 answer
201 views

BOW features classifying better than complex models like BERT

I am doing a document classification task and I find that using simple BOW features with a random forest provide better results than using complex models like BERT or ELECTRA even after doing some ...
Atirag's user avatar
  • 149
0 votes
1 answer
24 views

Transforming topics into text data

I was reading some articles on topic classification, in which some algorithm uses snippets of text as input and tries to classify them in topics, and I thought of implementing this technique in my ...
enzo's user avatar
  • 113
2 votes
3 answers
437 views

How can "word hashing" cause a collision in DSSM?

They say in their paper, that "word hashing" can cause a collision. But I don't understand, how. For example, if word good is tranformed to ...
Dims's user avatar
  • 412
1 vote
0 answers
201 views

Language Identification Better Results with Unigrams

I have a school project which consists of identifying each language of a tweet from a dataset of tweets. The dataset contains tweets in Spanish, Portuguese, English, Basque, Galician and Catalan. The ...
user avatar
1 vote
2 answers
512 views

Why CBOW model is called "continuous"?

The question is pretty clear from the Title itself, why the Continuous Bag of Words (CBOW) model is called continuous. I also don't know what exactly "distributed representation" of word vectors ...
Ruchit Patel's user avatar
2 votes
1 answer
908 views

could someone please give a concrete example to illustrate the Dirichlet distribution prior for bag-of-words?

I am aware of the notion of the Dirichlet distribution, a multivariate generalization of the beta distribution. To get parameters of the Dirichlet distribution prior for bag-of-words, this CMU ...
JJJohn's user avatar
  • 2,005
3 votes
2 answers
263 views

could someone please give an concrete example to illustrate what does Multiplicity mean in the context of Bag-of-words model?

This CMU Machine Learning Course is using the Bag-of-words model without too much explanation. wiki uses the term multiplicity to explain that model. The bag-of-words model is a simplifying ...
JJJohn's user avatar
  • 2,005
1 vote
2 answers
735 views

Text classification with small dataset for a specialized domain

I have a multiclass text classification problem where I have very few documents for each class. The classes are imbalanced but I want to be able to predict the class when I have at least 200 - 300 ...
nicnaz's user avatar
  • 77
1 vote
0 answers
71 views

Bag of Visual Words: is feature extraction even needed?

I'm currently implementing a BoVW as part of my lab project. The steps the algorithm used are as follows: spliting all photos into patches cluster these pathces using K-means based on pixel values of ...
Moran Reznik's user avatar
1 vote
1 answer
652 views

Classification using n-grams [closed]

I have $10000$ samples of 6-lettered strings of the following type Left                  Right  &...
bandit_king28's user avatar
1 vote
1 answer
22 views

Group of word representations

For word representation baseline people use bag-of-words or word embedding. Here, I want to understand all approaches that can be used for word representations. For example: -Bag-of-words (tfidf, n-...
aldin's user avatar
  • 11
3 votes
1 answer
2k views

Understanding the role of document size parameters in Latent Dirichlet Allocation

I am writing a pymc3-based implementation of Latent Dirichlet Allocation, and am referencing this CrossValidated answer (modified for pymc3) as well as pymc3's own tutorial on LDA, in addition to the ...
ely's user avatar
  • 2,352
7 votes
1 answer
3k views

Word2Vec : Difference between the two Weight matrices

In Word2Vec algorithm, two weight matrices are learnt : W : Input-hidden layer matrix W': Hidden-output layer matrix For reference, CBOW model architecture: Why is W chosen to represent the word ...
Journ's user avatar
  • 73
11 votes
5 answers
2k views

How to measure dispersion in word frequency data?

How can I quantify the amount of dispersion in a vector of word counts? I'm looking for a statistic that will be high for document A, because it contains many different words that occur infrequently, ...
dB''s user avatar
  • 317
1 vote
0 answers
82 views

What is the difference between training examples generated by continuous bag of words (CBOW) and skip-gram?

This is a simple question that is hard for me: Let's consider simple sentence A B C D and create training examples for skip-gram training (x, y) with number of ...
quester's user avatar
  • 498
1 vote
1 answer
436 views

Which classifiers do consider the order of the features?

In case the order of features can make a difference in the results of a classification approach, which classifier algorithms perform better? I know Naive Bayes/KNN use bag of words and ignore the ...
Sarah's user avatar
  • 123
3 votes
1 answer
1k views

Regularization in text classification with bag-of-words

I am performing a text categorization with bag of words and logistic regression. I have already heard about L1 and L2 regularization and used them for classification but with problems handling way ...
Elliot's user avatar
  • 203
1 vote
0 answers
412 views

Low score in sentiment analysis : how to increase it and maybe deal with class imbalance

It has been 2 weeks now I am working on SemEval task 4 (2016) : Sentiment Analysis on Twitter. The results I achieve are lower than what I expected for the three class classification problem : ...
Elliot's user avatar
  • 203
2 votes
1 answer
690 views

Find most similar sentence from one list of sentences to another

I have two lists of short sentences (List A and List B). For each short sentence in List A, I am trying to find the most similar short sentence in List B. Each list has a different count of elements ...
sloxton's user avatar
  • 21
1 vote
0 answers
365 views

Creation and validation of cluster for Bag of words

I recently came across a problem where I have been given a dataset of Bag of words, the description of the dataset is given in the readme file. What I have been trying to do is create clusters of ...
Sankalp Kotewar's user avatar
0 votes
0 answers
1k views

Combatting data sparsity, overfitting in bag of words model

I am looking at plots of learning curves (accuracy vs training examples) in order to compare different feature extraction methods that I am trying using a bag of words model (term presence vs. term ...
scmz's user avatar
  • 3
1 vote
1 answer
90 views

Is machine learning a viable approach to extract license references from source code files?

I am a complete newcomer to the field of machine learning. I do have a lot of experience in computer programming, but nothing related to ML. My question is whether or not ML would be a good approach ...
Travis's user avatar
  • 11
2 votes
1 answer
56 views

Searching for list of terms using Google in order to build a bag-of-words for a particular category [closed]

I am having a hard time understanding the process of building a bag-of-words. This will be a multiclass classification supervised machine learning problem wherein a webpage or a piece of text is ...
user avatar
3 votes
2 answers
453 views

Language Modelling using Neural Networks

I plan to make a Language Model in Python using Neural Networks. I've read that Neural Networks need vectors as input. One common vector representation in NLP is the Bag of Words model. Given a corpus ...
Jessie's user avatar
  • 53
4 votes
0 answers
321 views

Google gender-pay gap vs

Background: I read this: google schools US government about gender pay gap. It derives from this google blog post by Eileen Naughton, VP of People Operations. She asserts that google is somehow "...
EngrStudent's user avatar
  • 9,843
6 votes
1 answer
2k views

Understanding Word2Vec

I am trying to understand the word2vec algorithm (Mikolov et. al) but there are a few thing which I do not understand. I get that the activation from the input layer ot the hidden layer is linear and ...
Stefan Falk's user avatar
1 vote
1 answer
2k views

How does a bag-of-words model treat words that were never seen before (not in the training data)?

What happens when a text classifier using a bag-of-words model (let's say we're using logistic regression) encounters a word that the model has not seen before- aka, words that were not in the ...
Imu's user avatar
  • 113
2 votes
1 answer
326 views

Why do we need Tokenzier if we have Vectorizer

In the ML learning textbook I am working through, it says, that for NLP we construct a feature vector from the Text via the Bag of Words model. For that, we are using ...
Luca Thiede's user avatar
0 votes
1 answer
197 views

Calculate predictability of events over time

I'm trying to create a model / algorithm which learns the predictability of events over time, which takes into account both frequency and rarity. An example of what this could apply to (which is the ...
hizki's user avatar
  • 101
3 votes
2 answers
922 views

How to asses the optimal bag of words vector size?

I have a corpus with 6040592 words and 309074 types (different words). Knowing this information it is possible to know the optimal size of bag of words vectors in order to represent phrases? I am ...
alemol's user avatar
  • 141
1 vote
0 answers
61 views

Treating numerals/cardinals in Bag of Words (BOW) model

I wish to do topic modeling on text corpus some of which are about company earnings which has lots of numbers in it. It has no sentence structure. I think tagging numbers using nltk.pos_tagging can ...
pg2455's user avatar
  • 343