Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [bag-of-words]

A way of representing language data that consists of the constituent words w/ their individual frequencies. Ie, grammar & order, etc, are dropped to simplify the data.

0
votes
0answers
8 views

Why does all of NLP literature use Noise contrastive estimation loss for negative sampling instead of sampled softmax loss?

A sampled softmax function is like a regular softmax but randomly selects a given number of 'negative' samples. This is difference than NCE Loss, which doesn't use a softmax at all, it uses a ...
0
votes
0answers
7 views

Calculating the ratio of the (frequency of a specific word in the corpus/sum of the frequency of all the words)

I have the following code which gives me the list of top 10 words in a corpus in descending order of frequency: library(tidytext) tidybooks.nstop<-tidy_books %>% + anti_join(stop_words) ...
0
votes
0answers
19 views

Word embeddings NLP

Studying about word embeddings I have a few questions because I have been confused. According some textbooks we have two categories of word embeddings the sparse models which based on frequency (word ...
0
votes
0answers
34 views

why is negative sampling more often used in skip2gram rather than cbow

The first time I know negative sampling was from Skip2gram and the goal of negative sampling is to help the model not only learn what is correct but also what is wrong. But why negative sampling is ...
1
vote
0answers
22 views

Bag of Visual Words: is feature extraction even needed?

I'm currently implementing a BoVW as part of my lab project. The steps the algorithm used are as follows: spliting all photos into patches cluster these pathces using K-means based on pixel values of ...
0
votes
1answer
161 views

Classification using n-grams

I have $10000$ samples of 6-lettered strings of the following type Left                  Right  &...
1
vote
1answer
19 views

Group of word representations

For word representation baseline people use bag-of-words or word embedding. Here, I want to understand all approaches that can be used for word representations. For example: -Bag-of-words (tfidf, n-...
0
votes
0answers
206 views

TFIDF Weighting With Multiple Categorized Documents

I'm doing keyword extraction on about 300.000 documents. The documents are job advertisement. Im doing; 1- Preprosess the job description 2- Use sklearn tfidfVectorizer with min 1 max 3 ngrams ...
0
votes
0answers
204 views

Computing top n word pair co-occurrences from document term matrix

I used gensim to create a bag of words model. Although the output is much longer in reality, here is a snippet of the format outputted when creating a bag of words document-term matrix on the ...
2
votes
1answer
311 views

Understanding the role of document size parameters in Latent Dirichlet Allocation

I am writing a pymc3-based implementation of Latent Dirichlet Allocation, and am referencing this CrossValidated answer (modified for pymc3) as well as pymc3's own tutorial on LDA, in addition to the ...
4
votes
1answer
573 views

Word2Vec : Difference between the two Weight matrices

In Word2Vec algorithm, two weight matrices are learnt : W : Input-hidden layer matrix W': Hidden-output layer matrix For reference, CBOW model architecture: Why is W chosen to represent the word ...
10
votes
5answers
341 views

How to measure dispersion in word frequency data?

How can I quantify the amount of dispersion in a vector of word counts? I'm looking for a statistic that will be high for document A, because it contains many different words that occur infrequently, ...
1
vote
0answers
54 views

What is difference between training examples generated by continuous bag of words(CBOW) and skip-gram?

This is a simple question that is hard for me: Let's consider simple sentence A B C D and create training examples for skip-gram training (x, y) with number of ...
1
vote
1answer
121 views

Which classifiers do consider the order of the features?

In case the order of features can make a difference in the results of a classification approach, which classifier algorithms perform better? I know Naive Bayes/KNN use bag of words and ignore the ...
1
vote
1answer
520 views

Regularization in text classification with bag-of-words

I am performing a text categorization with bag of words and logistic regression. I have already heard about L1 and L2 regularization and used them for classification but with problems handling way ...
1
vote
0answers
205 views

Low score in sentiment analysis : how to increase it and maybe deal with class imbalance

It has been 2 weeks now I am working on SemEval task 4 (2016) : Sentiment Analysis on Twitter. The results I achieve are lower than what I expected for the three class classification problem : ...
1
vote
0answers
357 views

Find most similar sentence from one list of sentences to another

I have two lists of short sentences (List A and List B). For each short sentence in List A, I am trying to find the most similar short sentence in List B. Each list has a different count of elements ...
1
vote
0answers
294 views

Creation and validation of cluster for Bag of words

I recently came across a problem where I have been given a dataset of Bag of words, the description of the dataset is given in the readme file. What I have been trying to do is create clusters of ...
0
votes
0answers
373 views

Combatting data sparsity, overfitting in bag of words model

I am looking at plots of learning curves (accuracy vs training examples) in order to compare different feature extraction methods that I am trying using a bag of words model (term presence vs. term ...
1
vote
1answer
50 views

Is machine learning a viable approach to extract license references from source code files?

I am a complete newcomer to the field of machine learning. I do have a lot of experience in computer programming, but nothing related to ML. My question is whether or not ML would be a good approach ...
1
vote
1answer
46 views

Searching for list of terms using Google in order to build a bag-of-words for a particular category [closed]

I am having a hard time understanding the process of building a bag-of-words. This will be a multiclass classification supervised machine learning problem wherein a webpage or a piece of text is ...
3
votes
2answers
359 views

Language Modelling using Neural Networks

I plan to make a Language Model in Python using Neural Networks. I've read that Neural Networks need vectors as input. One common vector representation in NLP is the Bag of Words model. Given a corpus ...
5
votes
0answers
303 views

Google gender-pay gap vs

Background: I read this: google schools US government about gender pay gap. It derives from this google blog post by Eileen Naughton, VP of People Operations. She asserts that google is somehow "...
6
votes
1answer
938 views

Understanding Word2Vec

I am trying to understand the word2vec algorithm (Mikolov et. al) but there are a few thing which I do not understand. I get that the activation from the input layer ot the hidden layer is linear and ...
1
vote
1answer
807 views

How does a bag-of-words model treat words that were never seen before (not in the training data)?

What happens when a text classifier using a bag-of-words model (let's say we're using logistic regression) encounters a word that the model has not seen before- aka, words that were not in the ...
1
vote
1answer
89 views

Why do we need Tokenzier if we have Vectorizer

In the ML learning textbook I am working through, it says, that for NLP we construct a feature vector from the Text via the Bag of Words model. For that, we are using ...
0
votes
1answer
103 views

Calculate predictability of events over time

I'm trying to create a model / algorithm which learns the predictability of events over time, which takes into account both frequency and rarity. An example of what this could apply to (which is the ...
1
vote
0answers
60 views

Treating numerals/cardinals in Bag of Words (BOW) model

I wish to do topic modeling on text corpus some of which are about company earnings which has lots of numbers in it. It has no sentence structure. I think tagging numbers using nltk.pos_tagging can ...