Questions tagged [natural-language]

Natural Language Processing is a set of techniques from linguistics, artificial intelligence, machine learning and statistics that aim at processing and understanding human languages.

Filter by
Sorted by
Tagged with
1 vote
1 answer
24 views

Which metric to use for language translation?

So I am using a pre-trained model to do the language translation Eg: Input = "Good morning" Output = "Bonjour" I would like to see if the ...
user avatar
  • 71
0 votes
0 answers
20 views

Calculate a measure based on LDA topics and Hellinger distance

I am trying to calculate some sort of ambiguity/heterogeneity measure from text based topic probabilities from a Latent Dirichlet Allocation model and the Hellinger distance between the topic ...
user avatar
1 vote
1 answer
12 views

In a tranformer, the same word can have different attention weights in different sentences?

I'm trying to understand the transformer architecture for NLP. The main issue is regarding the attention weights. The same word can have different attention weights in different sentences, right?
user avatar
  • 378
0 votes
0 answers
8 views

How to determine distribution of word co-occurrences?

I'm trying to find most relevant noun phrases from abstracts of scientific articles. I tried following the methodology specified in the following paper: https://arxiv.org/pdf/1109.2058. I did the ...
user avatar
  • 168
1 vote
1 answer
19 views

BERT MLM - 80% [MASK], 10% random words and 10% same word - how does this work?

I have noticed that (from the original BERT paper) in the MLM training procedure, the authors decide to mask 15% of the words in a sentence. The mask works as following: The masked words are ...
user avatar
0 votes
0 answers
13 views

Possible ways to feed two variables containing text data into an ML model in an NLP problem

In a natural language processing (NLP) problem, we have a couple of variables, say A and B. A denotes a phrase (1-2 words) and B denotes another phrase (>3 words). There is one target variable that ...
user avatar
0 votes
0 answers
8 views

How to improve F1 score with LocalOutlierFactor

The classification we are working on is to predict the investment strategy the company is using by NLP, and there are four types of strategy. “Balanced Fund (Low Risk)”, “Fixed Income Long Only (Low ...
user avatar
0 votes
0 answers
15 views

How to detect Covariate shift of NLP models?

I have an NLP model, for example, Sentiment Analysis. This model serves in production. I want to detect Data Drift, and specifically Covariate Shift for this model. I saw that Cosine Similarity may ...
user avatar
1 vote
0 answers
19 views

Predict most probable document in a given set of documents by a given question

I would like to know / discuss which implementation would be the best solution to predict to a given question the document from a given set of documents which is has the highest probability of ...
user avatar
  • 11
0 votes
0 answers
7 views

Vector to sequence RNNs: do they take a random initial "prompt"?

I am going through the Deep Learning book by Ian Goodfellow (here) and came by the architecture for a vector to sequence RNN (Figure 10.9). I am not sure I understand how this architecture works and ...
user avatar
0 votes
0 answers
6 views

Text classification using BERT base model without fine-tuning

Let's say, I fine-tune a BERT base model, which I name MyBERT, for sentence classification (categorical classes) and I want to compare its performance with the BERT base model. But, BERT base model ...
user avatar
0 votes
0 answers
9 views

BERT frozen layers

What does it mean, conceptually, to 'freeze' some BERT layers e.g. when fine-tuning for text classification. I mean, BERT comes pre-trained and fine-tuning, through the classification head, adds some ...
user avatar
2 votes
1 answer
23 views

How to choose a sample of a text dataset to label?

I was reading about active learning recently and active learning seems to be used after the first model is generated. So, I was wandering if there are technics to choose what to label before ...
user avatar
0 votes
0 answers
8 views

How to track the dynamics of topics in a corpus with dates?

I am working with a set of news that span over a few weeks. For each day, I have approximately 10000 news articles on different topics. What I want to do is to track the topics of this news and how ...
user avatar
1 vote
0 answers
13 views

Extracting information from bills, tax statements, etc: What ML model to use?

I have a bunch of documents such as bank statements, utilities bills, personal expenditure invoices, etc. The document types range is very broad. Some of these files are saved as pictures, others as ...
user avatar
0 votes
0 answers
9 views

Hybrid approach for text categorization (Rule based + ML)

I want to build a multi-label text categorization system of paper abstracts with about 20 categories. For many of the categories keyword based logical rules exist with a low false positive and medium ...
user avatar
  • 1,119
0 votes
0 answers
9 views

Which NLP methods use gradient and activation methods?

I am doing a literature review of gradient-based methods for NLP. Yet, apart from linear and logistic regression, I have little knowledge of other methods using the gradient. So I have no knowledge of ...
user avatar
1 vote
1 answer
69 views

Should I remove stopwords before generating n-grams?

I'm wondering if the stopwords are useful in n-gram or it should be removed before generating n-gram. I would like to know best practices on extract features of text. I'm currently using nltk.
user avatar
  • 13
0 votes
1 answer
34 views

Inter Annotator Agreement for Question Answering

I would like to run an Inter Annotator Agreement (IAA) test for Question Answering. I've tried to look for a method to do it, but I wasn't able to get exactly what I need. I've read that there are ...
user avatar
0 votes
0 answers
11 views

Word embedding to support OOV words for identity embedding

I want to create a model that performs a user-id embedding (hash of a user) for a Graph Neural Network learning task, the problem I am facing is that I might have a very large corpus of users, which ...
user avatar
1 vote
0 answers
17 views

How to justify logarithmically scaled frequency for tf in tf-idf?

I am studying tf-idf (term frequency - inverse document frequency). The original logic for tf was straightforward: count of term t / number of total terms in the document. However, I came across the ...
user avatar
0 votes
0 answers
7 views

Return_State parameter in LSTM (Keras)

I'm a bit new to NLP, I've read multiple posts but couldn't find the intuitive purpose of setting return_state=True/False. I got answers like setting return_state will return cell output as well, this ...
user avatar
  • 101
0 votes
0 answers
16 views

How to design a method for finding multiword terms based on labeled data

Setup: I have many textdocuments that have been processed by an OCR Engine. These documents are Invoices and the endgoal is to classify words inside each document. If words on a document are seperated ...
user avatar
0 votes
0 answers
43 views

Using BERT Embeddings + Standard ML for text classification

I am trying to automatically detect whether a text is written by a Machine or a Human. My first approach was using a TF-IDF to build features for a logistic regression classifier, where I got an ...
user avatar
0 votes
0 answers
7 views

Suggestion regarding usage of pre-trained BERT

I have recently been working with the pre-trained BERT. It produces quite good results on supervised tasks with just a bit fine tuning. But now I wonder if I want to perform some unsupervised task on ...
user avatar
0 votes
0 answers
12 views

Different F-1 scores on dev and test set, more data doesn’t increase performance?

I am fine-tuning a BERT model on sentiment analysis for tweets in a particular language (this model was pre-trained on tweets too). Initially I had an imbalanced dataset of around 9k examples (80% ...
user avatar
  • 181
1 vote
0 answers
17 views

What is a word embedding approach that would work for these pre-labeled documents?

My Situation: I should start off with my end goal: I want to get a distance metric between each document and all of the other documents To get there, I first need to encode these topic labels so that ...
user avatar
1 vote
0 answers
7 views

Create Score with a skewed data

I'm trying to create a reputation score for sneakers using positive and negative sentiments of twitter on these sneakers (something like score = p+n). The problem is that the mean of positive != mean ...
user avatar
0 votes
1 answer
18 views

How to extract key words from text that predict the value of a metric?

I have a large data set of houses; per house a single string describing the interior of the house, and a separate score from 1 to 10 describing how well the house is actually renovated. I would like ...
user avatar
  • 23
0 votes
1 answer
13 views

Sequential model recommendation for text classification without deep learning?

I am doing a NLP project, I have about 500 lines of data and my goal is to have a binary classification model, if 1, then the input text is talking about a subject and if 0, the input text is not ...
user avatar
0 votes
0 answers
21 views

Which Gensim's LDA parameters to test for finding optimal model?

For now I want to test these parameters and their ranges: num_topics (10 to 500; otherwise process of training and Coherence computing gets too long) alpha: ...
user avatar
  • 1
1 vote
1 answer
19 views

How to handle out of the bag token in NLP?

In my current language model my model is unaware of any token that is out-of-bag for example:- In my summary generating model when we pass some token that is out-of-bag then my model will completely ...
user avatar
1 vote
0 answers
100 views

fine tune universal-sentence-encoder embeddings

I am new to NLP and Neural Networks. I want to do topic analysis for a dataset of reviews of our product. I tried to use the universal-sentence-encoder along with <...
user avatar
0 votes
1 answer
46 views

Normalizing Topic Vectors in Top2vec

I am trying to understand how Top2Vec works. I have some questions about the code that I could not find an answer for in the paper. A summary of what the algorithm does is that it: embeds words and ...
user avatar
1 vote
1 answer
16 views

How to extract numerical features that can separate well documents belonging to two different classes?

I have a group of texts belonging to two different classes. I would like to extract numerical features that can separate well the two classes. Right now I implemented a classic TF-IDF with a document ...
user avatar
0 votes
0 answers
15 views

Pretrained Language model comparison with binary sentiment classification

On two independent datasets, I am comparing XLNet and BERT models with binary sentiment classification tasks: the Twitter dataset, where sentences are short, and the IMDB review dataset, where ...
user avatar
0 votes
0 answers
6 views

How to decode neural consituency parser output

Background: I trained a neural consituency parser on PennTreeBank. As an input it will receive a tokenized sentence and as an output it will generate something like this: In: ...
user avatar
0 votes
1 answer
43 views

How to do a significance test on the performance of four classifiers on two different test sets?

I'm trying to find out if an authorship obfuscation technique leads to a statistically significant drop in authorship classification performance. To test this assumption, I have a dataset of texts ...
user avatar
  • 3
0 votes
0 answers
27 views

Continuous Bag of Words NY Time Corpus

I am working to implement the continuous bag of words approach on the New York Times corpus dataset. However, I am getting word embeddings that do not seem very useful based on a few examples of ...
user avatar
1 vote
1 answer
30 views

Language model gives different results when Bayes' theorem is applied

Say, the following example is our corpus: a quick brown fox jumps over the lazy dog. Here, there are 9 tokens in the corpus. ...
user avatar
  • 185
0 votes
0 answers
9 views

How to achieve good result in text classification when the data is very small?

I have a dataset with many user comments, I want to classify this dataset with label 0 or 1. The dataset only has 730 comments labeled as 0 and 65 labeled as 1. I have developed a simples model using ...
user avatar
0 votes
0 answers
22 views

When is the input of a transformer a matrix?

I have a question about the format of the input of the transformers, depicted in the image below: (taken from the [page](https:// blogs.oracle.com/datascience/multi-head-self-attention-in-nlp)). When ...
user avatar
  • 41
0 votes
0 answers
20 views

Using NER to classify messages based on keywords and rank them by importance

I am new to NLP. I am trying to classify a set of messages based on specific keywords based on importance of a user. For example, you would be given a list of messages and then a person would come and ...
user avatar
0 votes
0 answers
13 views

Calculating RNN loss (for a SINGLE example) as a sum of individual time step losses VS. an average of individual time step losses [duplicate]

In Andrew Ng's course, I see RNN loss being calculated as a sum of the losses from each time step as seen here: In Stanford's CS224N, I see loss calculated as an average of individual losses as seen ...
user avatar
  • 1
1 vote
0 answers
14 views

Log Naive Bayes NLP dropping the denominator [duplicate]

I'm learning about the the Naive Bayes classification and I don't get what the squiggly alpha sign means and what it means that "Denominator remains constant for given input." Is it because ...
user avatar
1 vote
0 answers
17 views

Softmax computation in Transformers

In word embedding model word2vec, computation of softmax is expensive process and hence we use many alternative as provided here. Prominently, ...
user avatar
  • 101
0 votes
0 answers
11 views

How can I make a text classification model that can understand the meaning of a word?

I am new to NLP and recently I have been working in a text classification model with 684 texts classified as 0 and only 77 classified as 1. My result so far is not bad, 74% precision and 77% recall. ...
user avatar
0 votes
0 answers
22 views

Classify products as fragile / non-fragile

I have a bunch of products and its data description. I don't have the labels for the products. I want to develop an algorithm to classify/cluster the products as Fragile / Non-Fragile. This is an ...
user avatar
1 vote
1 answer
28 views

Feature function for logistic regression

In his Natural Language Processing textbook, Eisenstein defines the logistic regression as follows: $$p(y|x,\theta) = \frac{\exp (\theta f(x,y))}{\sum_{y'} \exp (\theta f(x,y'))}$$ Where $\theta$ are ...
user avatar
  • 113
0 votes
1 answer
29 views

What is a better way to classify text using word2vec?

I am using word2vec to classify documents into various categories. Let's say we have a document: Thousands of people with student loan debt will have their debt ...
user avatar

1
2 3 4 5
22