Questions tagged [natural-language]

Natural Language Processing is a set of techniques from linguistics, artificial intelligence, machine learning and statistics that aim at processing and understanding human languages.

Filter by
Sorted by
Tagged with
-1
votes
0answers
6 views

How to deal with different vector dimensions when combining numerical categories and vectors generated from text (NLP)?

I have a dataset with categorical columns and one text column. As it is a Classification problem. I wanted to perform NLP based modelling on the text column and another ML model on the categorical ...
0
votes
1answer
15 views

For BERT-based models, do we absolutely have to include the [CLS] and [SEP] special tokens in the input data?

The thought just occurred to me while I was processing data. If we're using the [CLS] token for classification, then it would obviously make sense to include it, ...
0
votes
2answers
41 views

How to combine a Machine learning model and an NLP model? [closed]

I have a dataset with categorical columns and one text column. As it is a Classification problem. I wanted to perform NLP based modelling on the text column and another ML model on the categorical ...
0
votes
0answers
6 views

Fasttext and new character not seen in training

Trying to understand how fasttext handles seeing a character in a test environment that it never saw in training. Specifically, I’ve been given a dataset to train a model where someone removed all of ...
0
votes
0answers
12 views

NLP Classification over Taxonomy [closed]

I have a massive pool of data where each row maps a user-entered description to a code that classifies the mission that the description describes. I am trying to develop a model that at least helps ...
0
votes
0answers
31 views

frequency of a word that is close to itself in a list [closed]

I have 2 files where the word "test" is found 3 times each. distributed.txt this is only a test to see if test is same as test Another file has the word ...
0
votes
0answers
12 views

Mechanism behind word embedding [closed]

can someone explain to me how do we embedd words in a corpus, i have tried to read/watch tutorials talking about word embeddings but i just can not finish any of them, i get stuck on the passage from ...
0
votes
0answers
17 views

Is this a valid method to test textual difference?

For some research I am trying to reject/ not-reject the null-hypothesis that the proportion of a word is higher in a certain sample of a text than in the rest of the text. The full text is a ...
1
vote
1answer
23 views

How does Word2Vec CBOW softmax work with multiple context words?

I'm referring to following paper from Xin Rong - "word2vec Parameter Learning Explained", to be precise the equation (4): $$ p(w_j|w_I) = \frac{\exp(\mathbf{v’}^{T}_{w_{j}}\mathbf{v}_{w_{I}})...
0
votes
0answers
12 views

How do the trainable projection layer used in PRADO and pQRNN work?

Trainable projection layers are said to be a very powerful thing but after reading: https://www.aclweb.org/anthology/D19-1506.pdf https://arxiv.org/pdf/2101.08890.pdf I don't understand how it works....
0
votes
0answers
11 views

Calculate the sentiment knowing the single probabilities

I am working on finding the sentiment of some text. Right now I am using a python library called VADER that for each sentence it gives me back the probability that a specific phrase is positive, ...
0
votes
0answers
12 views

Sample size for doing topic modeling using LDA() in R (topicmodels package)

I just started to learn and do text analysis for open-text survey questions. My sample size is around 2000. I want to use the LDA() function in R (topicmodels package) to identify topics among the ...
0
votes
1answer
18 views

Best practice for named entity recognition on large texts

What are the best practices to apply NER to large texts (e.g 20 pages+)? One common advice is to split the text before passing it as input to the model. However this can require a significant manual ...
0
votes
0answers
4 views

Getting worse results when I indicate the class_weights using compute_class_weight while using a BERT Model

When using the compute_class_weight from sklearn, I'm getting worst results than when I didn't specify any weights at all when training a BERT Model. The reason why ...
1
vote
1answer
35 views

Why were/are LSTMs popular for so long for NLP?

Talking with a professor at uni about NLP methods, I asked why LSTMs (and any RNN variant for that matter) were so popular for so long, given the vanishing gradient problem. More specifically, ...
1
vote
0answers
19 views

How to parametrize a Decisiontree Classifier for NLP tasks?

I want to use sklearns DecisionTreeClassifier for sentiment analysis. I am fully aware that a decision tree is probably not a good model for this kind of task, but I want to try it anyways for the ...
1
vote
0answers
26 views

Rule of thumb for the minimum frequency for unknown words in a NLP Neural Network Language model?

I know there are approaches that process unknown words with their own embedding or process the unknown embedding with their own character neural model (e.g. char RNN or chat transformer). However, ...
1
vote
1answer
42 views

Deciding between Decoder-only or Encoder-only Transformers (BERT, GPT)

I just started learning about transformers and looked into the following 3 variants The original one from Attention Is All You Need (Encoder & Decoder) BERT (Encoder only) GPT-2 (Decoder only) ...
0
votes
0answers
5 views

Designing a Specialized Domain Text Summarizer

I am trying to develop an extractive summarizer for the medical domain. Here I am trying to tag the medical domain entities and classify them. I am trying to give a small example. Mr.XYZ has developed ...
0
votes
1answer
16 views

Sentiment Analysis but for Descriptions in Texts?

Reviewing literature about sentiment analysis, I can only find contributions focussing on the sentiment of the author. Famous examples are twitter posts or movie reviews. I wonder if there are ...
1
vote
1answer
13 views

Harmonic is used in F1 score because it is a conservative metric: How does it help being conservative?

I was reading Jurafsky 3rd edition, page 12-13 chapter 4 Can you explain why is it good to weigh more the smaller of the two items namely $\frac{1}{Precision}$ or $\frac{1}{Recall}$? Here is the link ...
0
votes
0answers
8 views

How to take the keywords from the given dataset to train GPT-2 based chatbot?

I am working with a dataset that contains Questions on various Events conducted by a college and the corresponding answers for the queries. I am using this dataset to train a GPT-2 355M model to ...
0
votes
0answers
23 views

Class token in ViT and BERT

I'm trying to understand the architecture of the ViT Paper, and noticed they use a CLASS token like in BERT. To the best of my understanding this token is used to gather knowledge of the entire class, ...
0
votes
0answers
11 views

Applying the activation function in this way is equivalent to the other way - RNN

I don't understand why in RNN the 2 following ways of applying the activation functions are equivalent: First way: $$ h_t = W\sigma(h_{t-1}) + U x_t + b $$ Second way: $$ g_t = \sigma(Wg_{t-1} + U x_t ...
0
votes
1answer
27 views

Generating embeddings for languages without a written representation?

I'm considering the topic of generating an NLP Embedding for a language without a written standard or a significant corpus. I realized that as challenging as that is, it is still not as challenging as ...
0
votes
1answer
9 views

Doc2vec Corpus Size Recommendation

I'm trying to make a semantic search engine with Doc2Vec where you query the model a document and it returns N most similar documents from its training corpus. I'm having trouble pushing accuracy past ...
0
votes
0answers
6 views

NER tagging schema for non contiguous tokens

Looking at NER tagging, it seems that the most common tagging procedure is IOB. But it seems that this kind of tagging is limited to cases where tokens from the ...
0
votes
0answers
14 views

Distance measure for hierarchical nominal data

I have categorical data which follow a hierarchical structure (in fact they're medical codes). For instance: C10: Diabetes Mellitus E00: Senile dementia E10: Schizophrenia E2B1: Chronic Depression G20:...
0
votes
0answers
15 views

Sentiment Analysis using an SVM - How to comprehend the models decision based on weights and input?

I am doing sentiment analysis using sklearns linear SVM and I want to try to figure out why certain texts get incorrectly classified by looking at the models weights and the vector for that text. For ...
0
votes
0answers
42 views

Word2Vec input implementation to Machine/Deep Learning

I am running an experiment using NLP. I am using Word2vec in order to have a distributed vector representation of the input text and then feed these representations in different Machine Learning (ML) ...
1
vote
1answer
33 views

Creating a question-answer chatbot

I was thinking of creating a chatbot for providing info about my college as a side-project, which aims to answer questions like "How many swimming pools are there?" to "How many lecture ...
0
votes
0answers
11 views

How should data be organized for topic modelling

Say I have 10 documents with 10 sentences each. I'm curious how should my raw data look like? Is there a standard? Should it be like? ...
0
votes
0answers
20 views

Perplexity Score for Topic modelling

I am new to topic modelling using LDA. I build a initial version of topic modelling. I tried different topic models. I got to know that perplexity score is a good measure for evaluating topic models. ...
0
votes
0answers
35 views

Neural network overfitting from more samples

I am fitting a neural network model for a text classification task, the issue is that as I introduce more samples from the dataset I start to see more overfitting. This charts show train/val accuracy ...
2
votes
0answers
16 views

Do transformers need cross-validation

I am trying to train a NLP text classification model using HuggingFace's Transformer package. I have noticed that they do not provide a cross validation function and couldn't find any example of ...
0
votes
0answers
3 views

Captions are better, yet validation loss is increasing

I am training and validating an image captioning model with the following architecture: Encoder: ResNet-101 Pre-trained on ImageNet Decoder: GRU (1-Layer) Embeddings: Last BERT hidden state I ...
1
vote
0answers
20 views

Interpretation of negative values in the results of LSA

How can I interpret the results of LSA? From the following table, I can understand that documents 0 and 1 (dealing with cats) fall into Topic 2. Document 4, which talks about both dog and cat, falls ...
3
votes
1answer
34 views

Do we use maximum likelihood or cross entropy Loss for training skip-gram model?

In the skip gram model, maximising the likelihood of the context words given the middle word is equivalent to minimising the objective function $J(\theta)$, where $$J(\theta) = -\frac{1}{T}\sum_{t=1}^...
0
votes
0answers
10 views

How to find the outliers(trending data) for real estate data

I have prices, location, type, rooms, etc info about various houses from a real estate website. I want to find the trending data or the houses which are unique, e.g. a house in New York, with 2 rooms ...
0
votes
0answers
13 views

Suppressing false alarms with capturing information from unstructured corpus

In our team, we had previously deployed a machine learning anomaly detection tool at a chemical plant. It has been observed in certain cases that ongoing manual operations/interventions at plant can ...
0
votes
0answers
25 views

Information Retrieval and Event Prediction from Unstructured Document Corpus

My question is quite open ended. In some chemical plant, by using the sensor data available we first deployed a machine learning tool that can predict the onset on anomalous behavior with some decent ...
0
votes
0answers
21 views

Using CTC loss to compare strings: Unexpected Behaviour

Out of curiosity, I tried to use CTC loss(Pytorch implementation) to compare two strings instead of a conditional probability distribution and a string. I followed the following steps to convert the ...
3
votes
1answer
69 views

Viterbi Algorithm - Most likely sequence vs sequence of most likely states

I'm trying to understand why the following pseudo-code function is correct: ...
0
votes
0answers
12 views

Statistical test for two sets of word embedding vectors being 'similar enough'

I have two models which, for given input query q, output 'k' similar sentences from the same corpus. The i-th element of the output is a vector-embedding representation of the i-th most similar ...
2
votes
1answer
103 views

Are text generation models generative or discriminative?

I've recently been studying generative and discriminative models, and I had a question regarding text generation. I'm aware that generative models model $P(X, Y)$ and discriminative models model $P(Y |...
1
vote
0answers
21 views

Skipgram model theory confusion

In the output layer of a skipgram model, there are $|\text{Context}|*|\text{Vocab}|$ values. And for each context word, the values are basically the dot product of the input word representation and ...
1
vote
0answers
25 views

How should I go about recoding open-ended answers from a survey?

I have some open-ended survey data that I'm trying to recode, but the range of answers is very large (e.g. one question got responses of 'word', 'separate', 'mesabatainia', and 'abra cadabra alakazam')...
0
votes
0answers
14 views

SparkNLP, Tensorflow, Pytorch for NLP

I used Tensorflow, Keras and Pytorch in the past for NLP related works. New company uses Hadoop, which I only have basic concepts. I looked around and found the NLP package associated with the ...
1
vote
0answers
12 views

What is the marginal log-likelihood in a Multi-Head Model?

I have to study the model described here. Given a passage text and a question about it, the model tries to find the correct answer to the question. This model follows a multi-head approach: each head ...
0
votes
0answers
7 views

How good is zero-shot learning for few-shot/low-shot learning?

I currently face a few-shot/low-shot learning problem in NLP domain. This problem is just like the definition of the problem itself: some labels appears very infrequently. Based on my experiments, ...

1
2 3 4 5
21