Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [natural-language]

Natural Language Processing is a set of techniques from linguistics, artificial intelligence, machine learning and statistics that aim at processing and understanding human languages.

0
votes
0answers
3 views

Why does HMM model for POS tagger works better with less data

The following is the result of using an HMM model for POS tagging a corpus. The following shows the size of training data and the precision on 1000 words test data: ...
0
votes
0answers
9 views

What does a word embedding's dimension signify?

I'm currently studying NLP and had a question regarding word embeddings. My understanding of a word embedding is that it is, simply put, a modular way of expressing words and phrases as vector ...
0
votes
0answers
3 views

How does short-term dependency improve performance for NLP models?

I was reading a paper titled Sequence to Sequence Learning with Neural Networks (Sutskevers, Vinyals, and Le - 2014 NIPS) and had a question regarding the concepts of "short-term dependency" and "long-...
0
votes
0answers
7 views

using latent dirichlet allocation to reduce the number of dimensions in bag of words model?

Does anyone have experience reducing the dimensions in a traditional bag of words model? For example, if you want to train a decision tree on a large set of reviews, the size of the vocabulary ...
0
votes
0answers
7 views

Is pooling acceptable to evaluate information extraction?

When dealing with information extraction of unbalanced classes (e.g. the desired class has a prevalence of 0.5%), the required sample size for validation might be huge (thousands of cases and more), ...
0
votes
1answer
37 views

A model (neural network) for sets of arbitrary length [on hold]

I've been searching for a model that is close to RNN (is well suited for investigating sets of arbitrary lengths) but is insensitive to order. I'm aware of bidirectional RNNs. I've also found a 'bag ...
0
votes
0answers
14 views

Why does 4-gram work better than trigram or bigram or unigram in my experiments?

In a binary classification task, I used Logistic regression, decision tree and Adaboost with decision tree (max_depth=1). For each of the machine learning task, I used GridSearchCV to choose the ...
0
votes
0answers
12 views

Find Trends and Behavioral insights from Twitter data [closed]

I have over 200k tweets in csv format and I want to find music preferences, behavioral insights and some patterns, trends from this corpus. Iam comfortable with python. I dont want to just do the ...
0
votes
0answers
20 views

BERT Classification fine tuning for Q & A

I want to fine-tune BERT for Q & A in a different way than the SQuAD mission: I have pairs of (question, answer) Part of them are the correct answer (Label - 1) Part of them are the incorrect ...
0
votes
0answers
15 views

How to Calculate Perplexity or cross entropy from Probability Distribution for a certain sentence?

I have a list of words starting with the letter "s" and their frequency count. From this, I'm trying to build a language model. I don't have the whole text, so I can't do the conditional probability, ...
0
votes
0answers
13 views

Find categories of question using a pre-defined dictionary

I want to create a function that will automatically suggests categories when a user input a questions. The first step that I have done is to create a pre-defined dictionary with keywords and values (...
0
votes
0answers
7 views

Interpretation of NLP pipeline for topic discovery using gaussian mixture model clustering

I built a pipeline that does the following to discover topics out of a (very big: 50k docs per ~350 terms) Term Document Matrix: Compute the TfIdf score for each Term x Document pair; Rescale each ...
0
votes
1answer
12 views

Interpretation of the following logistic regression problem

I have a function that gives the probability of Y=1 given X i.e P(Y=1|X)=f(wX). This function is dependent on variables w and X and I have to give the range of w ...
0
votes
0answers
16 views

Efficiently normalize word embeddings

I'm using glove word embedding and would like to [-1,1] normalize it using python. The data is in the format of a dict with the word as key and a ...
0
votes
0answers
21 views

Accuracy of RNN getting stuck after 90% [duplicate]

I am using Keras RNN Cell to perform parts of speech tagging. The architecture is as follows(I cannot put the code because of privacy reasons) : An embedding layer of of 40 units of shape (...
0
votes
1answer
11 views

How does one use Convolutional Neural Nets (CNNs) on varying size sentences for NLP so that the final fully connected layer can remain fixed size?

I wanted to use CNNs to classify sentences. The sentences are varying length. I am going to use standard Word Embeddings (any sort of pre-trained vectors) as features for each word then concatenate ...
0
votes
0answers
11 views

NLP - how do you randomly draw negative samples?

From my understanding, negative sampling randomly samples K negative samples from a noise distribution, P(w). The noise ...
0
votes
0answers
9 views

Pointwise Mutual Information using spacy or just detailed explaination

So, I have been trying to play around with NLP recently and decided to work on a project involving Emotional Analysis. I have been following this particular research, http://www.cse.yorku.ca/~aan/...
1
vote
1answer
18 views

Having trouble figuring out how loss was calculated for SQuAD task in BERT paper

The BERT Paper https://arxiv.org/pdf/1810.04805.pdf Section 4.2 covers the SQuAD training. So from my understanding, there are two extra parameters trained, they are two vectors with the same ...
0
votes
0answers
13 views

What kind of errors could be responsible for low loss but disastrous BLEU in neural machine translation?

Please notice: I originally asked this question in Stackoverflow but I have been asked to move it here. I'm working on a custom neural machine translation model with the Fairseq framework (PyTorch), ...
0
votes
0answers
5 views

Implementation of Traditional Language Models

Problem I am now reading this paper (A Bit of Progress in Language Modeling) to know language modeling techniques prior to methods based on neural network. However, since this paper is rather old (...
0
votes
0answers
14 views

Are these Multi-label document classification experiment steps sensible?

I plan to filter an input document using 4 different labels. Just for an example, a document discussing about movie summary needs to be labeled with 4 labels (Romance, Drama, Fiction, Hollywood). ...
0
votes
2answers
28 views

Measuring similarity between document and category

Lets say I have a word embedding model, a set of documents and N categories. Lets say the categories are "cars" and "planes". I want to categorize the documents as either being about cars or planes. ...
0
votes
0answers
6 views

What would be the biggest considerations in using an SVM for NLP?

Evaluating a linear SVM on an NLP corpus where there are 150,000 data examples but each language sample is reasonably short(10-15 words). This is evaluated against a code that is a topic. For example "...
0
votes
0answers
10 views

Term x term matrix for text clustering. What to do with subterms of n-grams

I am doing topic discovery on a large corpus. Reading here and there I found some papers saying that in case of big, sparse document x term matrices is better to create first a term x term similarity ...
2
votes
4answers
95 views

Text Embeddings on a Small Dataset

I am trying to solve a binary text classification problem of academic text in a niche domain (Generative vs Cognitive Linguistics). My target text data consists of near 400 paper abstracts with less ...
1
vote
1answer
32 views

Model Decreasing in Accuracy With More Training Data

I am training a tagger to predict whether or not a "word" is a proper noun or not. To do this I take in a list of "words" and their tags for part of speech. I then change all tags that aren't the tag ...
0
votes
0answers
35 views

How to use the Expectation Maximization (EM) algorithm for Part of Speech (POS) tagging?

I want to know how can we use the EM algorithm for Part of Speech (POS) tagging. The data is a set of sentences Xs and their POS tags Ys i.e. a sentence X is a sequence of words $(X_1,X_2,\ldots, ...
0
votes
0answers
5 views

How PV-DBOW works

The authors of the Paragraph Vector paper describe PV-DBOW with: 2.3. Paragraph Vector without word ordering: Distributed bag of words The above method considers the concatenation of the ...
0
votes
0answers
26 views

Kolmogorov–Smirnov test on text data

The Kolmogorov–Smirnov test a very efficient way to determine if two samples are significantly different from each other or whether the CDF between two different samples fit each other. This can be ...
0
votes
0answers
8 views

Why does all of NLP literature use Noise contrastive estimation loss for negative sampling instead of sampled softmax loss?

A sampled softmax function is like a regular softmax but randomly selects a given number of 'negative' samples. This is difference than NCE Loss, which doesn't use a softmax at all, it uses a ...
0
votes
0answers
6 views

contextual embedding algorithm including continuous variables

Is there any work that allows contextual embedding of events that allows not only for categorical information but also some magnitude information (as opposed to word2vec, GloVe etc) I have a series ...
0
votes
0answers
21 views

How to statistically infer common pattern in text

Am trying to solve a problem where I need to infer common patterns in text for example, the data below, with bare eyes it can be noticed there is a pattern and that is ...
0
votes
0answers
18 views

Should the <s> and <e> be included in the vocabulary while calculating probability of a sentence in a Bigram model with Laplace smoothing?

I am working through an example of Add-1 smoothing in the context of NLP Say that there is the following corpus (start and end tokens included) ...
0
votes
0answers
13 views

Assigning weights to the features used in content based recommendation

I am trying to make a recommendation engine for book business which has following features associated with the books: Book Region Book Market Segment Publish Date Book Genre Book Type and so on ...
0
votes
0answers
15 views

How we handle unknown bigrams in bigram probability model with Good-Turing discounting?

Assume Good-Turing discounting. Assume number of unknown words is equal to the number of known words in our event space. Let $s$ be a sentece such that $s=w_1,w_2,\ldots,w_n$. We know the probability ...
1
vote
2answers
16 views

Python - Concept Extraction and Searching Algorithms for Document Corpus

I have a document corpus which I would like to query to find documents which have the same or similar concepts. This is different from a keyword search in that the term would not need to appear in ...
0
votes
0answers
12 views

How should I understand a Self-Organizing Fuzzy Neural Network?

I'm currently doing research to write a paper for a conference submission (undergraduate-level) and had a question regarding the research I've been conducting. My topic is on using Twitter sentiment ...
0
votes
0answers
25 views

performing function (maxcol) across the column

I'm currently going through this paper: Bidirectional Attention Flow for Machine Comprehension, Seo, Minjoon, et al. (2016) They perform a $max_{col}$ function over a matrix $S \in \mathbb R^{TxJ}$: ...
1
vote
0answers
28 views

Text document clustering using community detection algorithms

I have a corpus of documents. I want to do clustering of similar documents by using community discovery algorithm. Initially I preprocessed the corpus by using nltk. Then each document is converted ...
0
votes
0answers
71 views

Python + Machine Learning : string matching problem

I have been given one problem to solve: The problem is explained below: The company maintains a dataset for specifications of all the products (nearly 4,500 at present) which it sells. Now each ...
1
vote
1answer
28 views

Summary statitics to describe topic x term distribution in NLP

I created a topic model which outputted 11 topics out of 437 terms on ~60000 small documents. I wanted to show how good each topic is. But I don't know what "good" means in this case. Here's the ...
2
votes
1answer
45 views

How to infer one-to-one/one-to-many relationship? [closed]

We have a file with IP addresses patterns as shown below: ...
0
votes
0answers
8 views

Paragraph breakdown in natural language processing

I couldn't find an NLP stack exchange site, so I'm hoping this is the right place to post this type of question! I'm learning about NLP and have a reasonable grasp of tokenisation, parsing sentences ...
0
votes
0answers
16 views

Clustering positive and negative qualifiers with word2vec

I am looking to find whether a potential qualifier is positive, negative, or unknown. Example positive qualifiers are: increase, positive, raise. Example negative qualifiers are: decrease, negative, ...
0
votes
1answer
24 views

Encoding Layers in the Transformer

In the transformer architecture for NLP, at each layer there are multiple self-attention filters. My question is about the encoded content within these filters. An example can be found here: My ...
0
votes
0answers
10 views

Influence of neighbouring characters on a given position

I have 10,000 fixed length strings of DNA sequences. An example would be ATTGGGT M GCGGCTG. Now the character marked M is a position of interest to me (say something that causes diseases). ...
0
votes
1answer
19 views

Correct algorithm for string classification

I have a long list of DNA strings (of equal length) made of 4 letters (A,T,G,C). I want to do a binary classification on the strings. I have two basic quetsions: I have a lot of duplicate strings ...
2
votes
0answers
27 views

What is the number of filter when using CNN for sentence classification

I am new to machine learning and NLP. During reading convolutional neural networks for sentence classification I'm having trouble understanding it. In the paper it says that a feature map c has ...
0
votes
0answers
19 views

Word embeddings NLP

Studying about word embeddings I have a few questions because I have been confused. According some textbooks we have two categories of word embeddings the sparse models which based on frequency (word ...