Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [natural-language]

Natural Language Processing is a set of techniques from linguistics, artificial intelligence, machine learning and statistics that aim at processing and understanding human languages.

0
votes
0answers
3 views

How we handle unknown bigrams in bigram probability model with Good-Turing discounting?

Assume Good-Turing discounting. Assume number of unknown words is equal to the number of known words in our event space. Let $s$ be a sentece such that $s=w_1,w_2,\ldots,w_n$. We know the probability ...
1
vote
2answers
13 views

Python - Concept Extraction and Searching Algorithms for Document Corpus

I have a document corpus which I would like to query to find documents which have the same or similar concepts. This is different from a keyword search in that the term would not need to appear in ...
0
votes
0answers
7 views

How should I understand a Self-Organizing Fuzzy Neural Network?

I'm currently doing research to write a paper for a conference submission (undergraduate-level) and had a question regarding the research I've been conducting. My topic is on using Twitter sentiment ...
0
votes
0answers
18 views

performing function (maxcol) across the column

I'm currently going through this paper: Bidirectional Attention Flow for Machine Comprehension, Seo, Minjoon, et al. (2016) They perform a $max_{col}$ function over a matrix $S \in \mathbb R^{TxJ}$: ...
1
vote
0answers
10 views

Text document clustering using community detection algorithms

I have a corpus of documents. I want to do clustering of similar documents by using community discovery algorithm. Initially I preprocessed the corpus by using nltk. Then each document is converted ...
0
votes
0answers
31 views

Python + Machine Learning : string matching problem

I have been given one problem to solve: The problem is explained below: The company maintains a dataset for specifications of all the products (nearly 4,500 at present) which it sells. Now each ...
0
votes
0answers
5 views

Summary statitics to describe topic x term distribution in NLP

I created a topic model which outputted 11 topics out of 437 terms on ~60000 small documents. I wanted to show how good each topic is. But I don't know what "good" means in this case. Here's the ...
2
votes
1answer
37 views

How to infer one-to-one/one-to-many relationship? [on hold]

We have a file with IP addresses patterns as shown below: ...
0
votes
0answers
40 views

Text translation in python without connecting to web? [closed]

I am trying to translate Chinese text to English in python locally. I have tried following libraries. py-translate googletrans translate etc. But they connect to some APIs for translation. I am ...
0
votes
0answers
8 views

Paragraph breakdown in natural language processing

I couldn't find an NLP stack exchange site, so I'm hoping this is the right place to post this type of question! I'm learning about NLP and have a reasonable grasp of tokenisation, parsing sentences ...
0
votes
0answers
9 views

Clustering positive and negative qualifiers with word2vec

I am looking to find whether a potential qualifier is positive, negative, or unknown. Example positive qualifiers are: increase, positive, raise. Example negative qualifiers are: decrease, negative, ...
-1
votes
0answers
21 views

Identifying passwords with Machine Learning [closed]

I'm developing a mechanism to identify passwords, secrets (API keys etc.) which have been posted to an internal wiki. The dataset consists of around 1 million pages, consisting of guides, policies ...
0
votes
1answer
14 views

Encoding Layers in the Transformer

In the transformer architecture for NLP, at each layer there are multiple self-attention filters. My question is about the encoded content within these filters. An example can be found here: My ...
0
votes
0answers
9 views

Influence of neighbouring characters on a given position

I have 10,000 fixed length strings of DNA sequences. An example would be ATTGGGT M GCGGCTG. Now the character marked M is a position of interest to me (say something that causes diseases). ...
0
votes
1answer
19 views

Correct algorithm for string classification

I have a long list of DNA strings (of equal length) made of 4 letters (A,T,G,C). I want to do a binary classification on the strings. I have two basic quetsions: I have a lot of duplicate strings ...
2
votes
0answers
23 views

What is the number of filter when using CNN for sentence classification

I am new to machine learning and NLP. During reading convolutional neural networks for sentence classification I'm having trouble understanding it. In the paper it says that a feature map c has ...
0
votes
0answers
18 views

Word embeddings NLP

Studying about word embeddings I have a few questions because I have been confused. According some textbooks we have two categories of word embeddings the sparse models which based on frequency (word ...
0
votes
0answers
15 views

Guided LDA to categorize software requirements

I'm developing a application to categorize requirements in a requirement specification in to categories like database, front end, back end, etc. So for that I'm trying to use Guided LDA since labelled ...
0
votes
0answers
3 views

How to deal with inconsistent pretrained word embedding and expected hidden layer size

I'm just new to use pre-trained embeddings, currently trying to build a Transformer model with GloVe pre-trained word embedding. Yet since the GloVe only provides embedding with dimension 100/200/300 ...
0
votes
0answers
20 views

Attention for short sequence length. Is it reasonable?

Will the attention mechanism be useful for the short sequence length? Let's say your training corpus has each query of MAX length 10. and most queries are of word length 3-4 words. How reasonable is ...
0
votes
0answers
13 views

Combining the topics of two Latent Dirichlet Allocations

How to combine two LDAs? Let's suppose we have one corpus and we have estimated two Latent Dirichlet Allocations for two sub-corpuses : one for sub corpus A and another for sub corpus B. Now first ...
0
votes
0answers
13 views

Visualize Named Entities Relations

I have a dataset of named entities (NER) and text documents. For each entity I have N documents associated, while an entity belongs to one among multiple classes like (here the CoreNLP ones) ...
2
votes
1answer
30 views

Machine learning for product names

I have a machine learning challenge I may be over thinking. I have a set of 3.5 million products (not unique, there are multiple instances of each product). Each product has a "description" from it's ...
0
votes
0answers
22 views

Huge difference between training/testing accuracies

I'm working in a kind of a sentiment classification (binary) task. Using google's pre-trained word2vec vectors for the embedding layer (tried other word vectors as well) and 2 Convolutional layers for ...
0
votes
1answer
12 views

What is the motivation to train one's own word embedding model?

I've been using a few big word embedding models like word2vec & FastText, and they work very well on most problems. I am now adressing a new kind of data, on which they perform quite poorly, and I ...
2
votes
1answer
30 views

Word Embedding for Sentiment Analysis

I am working on sentiment analysis of text. I am using keras word embedding. If my embedding has a vocabulary of 50 and an input length of 4 and I choose an embedding space of 8 dimensions, how will ...
0
votes
0answers
15 views

Can an embedding layer be replaced by a fully connected layer?

Due to architecture choices and organization of code, I have a file called data.py that processes texts and returns two vectors : X and Y which are the vectorized text and the corresponding label. ...
2
votes
0answers
34 views

Is there a well established algorithm to match two documents on a semantic level?

I have a set of documents from a wide variety of topics and I would like to retrieve the ones that are more similar to a new document provided. A search based on common words is not good enough, so ...
1
vote
1answer
34 views

What problem is it when I want to map these documents to these 3 different labels?

I am completely new to machine learning, and that means I am new to the ML-related jargon too. I have a problem at hand where there about a 1000 documents (on an average 500 words each) which need to ...
0
votes
0answers
11 views

GridSearchCV with one-hot y: prediction yields 1-dim array

I run a classification by means of a neural network, thus my y-values are converted to a one-hot matrix: ...
1
vote
1answer
42 views

Understand the output layer of transformer

I'm trying to understand transformer from the paper, attention is all you need. I'm puzzled by the last piece on the linear -> softmax block on the decoder output, and wasn't able to find more ...
0
votes
0answers
10 views

Caret: Feature selection with Chi2 / f_classif

I try to classify texts which I have converted to term-document matrices before. I would like to perform feature selection to reduce the number of predictors. In Python, you can do this by means of ...
1
vote
1answer
10 views

Good way to use word similarity as a feature in supervised ML on text

I have a pretty low N data set of small sentences tagged with a label. I would like to create a classifier on this dataset. The word choice is not very variable since the domain is pretty specific. ...
0
votes
0answers
16 views

How to train a RNN language model?

I want to train a RNN-based language model from https://arxiv.org/pdf/1409.2329.pdf for next word prediction. How to split the sentences from the dataset into input and ground truth during the ...
0
votes
1answer
63 views

how does the loss function work in word2vec?

I was watching CS224n and I Came across this equation for word2vec loss function. As in the blue box, "for each document\training example t we are calculating the probability of context words given ...
2
votes
1answer
26 views

Natural Language Processing: Basic Dimension Reduction with SVD of a Co-Occurence Matrix

Given sentences I enjoy flying. I like NLP. I like deep learning We can form a Co-Occurrence Matrix as follows: Now we can apply Singular Value Decomposition to this matrix to get $X = U \Sigma V^...
0
votes
1answer
26 views

Difference between ELMo and a normal deep network

I was reading up about ELMo and what I could gather was that we essentially combine the weights from different stacked lstm/gru layers for a given token as different layers are suited for different ...
1
vote
1answer
34 views

Finding similar text - algorithms and evaluation

I've been asked to create a program that will rank similar texts to an input text given a collection of text. So far I've been using a tdidf representation and cosine similarity with a lot of regex-...
0
votes
1answer
19 views

Likelihood for a test data (sequence of characters) given two unigram models

I would like to find the likelihood of a sequence of characters (the test data), given two unigram models. The sequence (test data) is: A B C B B The models ...
0
votes
0answers
20 views

Architectures for Text Genre Classification

I am currently trying to build a model for giving genres to news articles. I was wondering what kind of architectures would be good to use for such a task? I am pretty unfamiliar with the current ...
0
votes
0answers
15 views

Weird prediction with binary classification on unseen textual data

I try to solve a problem which looks really simple. However I meet an obstacle and get stuck. I have a corpus of texts. I have to assign 0 to 1 to them (appropriate or not). There are a lot of ...
0
votes
0answers
26 views

What is the maximum number of features in Logistic Regression Problem

I was doing Text classification(binary) hosted on kaggle with approx 1.3 millions observations. My approach is to use Logistic Regression after computing the TF-IDF matrix with n-grams = 1:3. With ...
0
votes
1answer
18 views

How to understand “multimodal” RNNs for image captioning?

This paper Deep Visual-Semantic Alignments for Generating Image Descriptions on image captioning proposed a Multimodal Recurrent Neural Network architecture. From my understanding, the multimodal RNN ...
0
votes
0answers
11 views

Sparsity issue in co-occurance matrix

I am a noob in NLP. I am trying to put together a matrix for the occurrence count for pair of special characters in each paper. The special characters are defined in a list. I will have some special ...
0
votes
0answers
21 views

Getting the document-per-topic loading using TextmineR package by passing term co-occurrence matrix

I am using TextmineR package to find the most similar documents to given document list. I used the following code to generate the tcm not dtm ...
0
votes
2answers
45 views

How to use last predicted value as feature? NLP NER mission

I'm performing NER (Named entity recognition) For example: Seq: When Donald Trump announced... Tags: O B-Person L-Person O When I'm predicting ...
0
votes
0answers
39 views

LSTMs and Opening/Closing Brackets

I'm training a character-level LSTM to generate molecules using the SMILES system. Each molecule is represented as a string of characters, looking something like this: Cn1c(Nc2c(Cl)ccc(CNC(=O)C(C)(C)...
0
votes
0answers
11 views

Classifying company with financial transaction data

I am trying to classify a table of financial transaction data by company, but the data I will be using as a classifier can have a lot of variation from observation to observation for transactions that ...
0
votes
1answer
39 views

Which model should I try first?

I think about appropriate modelling technique in the following task: I have news texts (around 50K), and I have news topics made from the texts (250) which have various number of texts that made them ...
0
votes
0answers
11 views

Modeling words in a language based on their characters

I have different sets of strings, where I assume that each set follows some rules or patterns. For example, the first character must be a number, or the 3rd and the last characters must be the same, ...