Questions tagged [nltk]

NLTK stands for Natural Language ToolKit, a Python-based platform for working with human language data.

Filter by
Sorted by
Tagged with
0
votes
0answers
9 views

Bert service python getting word embeddings efficiently [closed]

I have a list of 5000 words and I need to get their word embeddings using bert service. I use the command ...
0
votes
0answers
39 views

is it possible to form precision-recall curve using one test dataset for my algorithm? [duplicate]

I'm working on knowledge graph, more precisely in natural language processing field. To evaluate the components of my algorithm, it is necessary to be able to classify the good and the poor ...
0
votes
0answers
19 views

What Multi-Class Text Classification alghoritm return few possible option?

I'm looking for some alghoritm to use for Multi-Class Text Classification and returns not only class with the highest possibility but a few ones. Its like with gensim Word2Vec: ...
1
vote
1answer
253 views

nltk multi_kappa (Davies and Fleiss) or alpha (Krippendorff)?

I'm using inter-rater agreement to evaluate the agreement in my rating dataset. I have a set of N examples distributed among M ...
0
votes
0answers
298 views

NLTK BigramAssocMeasures.pmi is give same score for all bigrams

I am trying to use BigramAssocMeasures PMI to find the most import bigrams however it's giving all Bigrams the same score, so I end up with a list in alphabetical order when I use ...
0
votes
1answer
956 views

NLP various probabilities estimators in nltk

I saw there are many types of probabilities in nltk: MLE, ELE, Laplace, Heldout, KnereserNey, Lidstone, Random, WittenBel.. What is the exact difference between them and when should I use each? My ...
0
votes
0answers
55 views

Naive Bayes Assignment of Feature Probability

I'm using the .show_most_informative_features() function from NLTK's Naive Bayes to generate features to be used with a lexicon. In the case of my binary-classification problem, these features are ...
0
votes
0answers
146 views

Handling False Positive in a Classifier

Suppose I have the following code of an NLTK Naive Bayes Classifier. It is a toy example of a sentiment analysis implementation. ...
1
vote
0answers
17 views

How to proceed with feature extraction?

I am building a multi class text classifier , that has data set of a job portal. The data set consists of names of organisations mapped to actual name (see below). I want to make a ml model which can ...
13
votes
2answers
8k views

How is the .similarity method in SpaCy computed?

Not Sure if this is the right stack site, but here goes. How does the .similiarity method work? Wow spaCy is great! Its tfidf model could be easier, but w2v with only one line of code?! In his 10 ...
0
votes
1answer
67 views

How to classify text when having very little training data

I have a dataframe as follows: New_Text | New_Score review1 | Positive review2 | Negative review4 | Positive ... and so on. I want to create a model ...
0
votes
0answers
331 views

Sentiment Analysis - Emoticon

I am working on an NLP project, my objective is to compute a Sentiment Analysis over short text message. So far, I did not come up with a solution to account the emoticons. I want to use a machine ...
1
vote
2answers
56 views

Why does my sentiment analyser only output 1 label?

I am trying to do sentiment analysis on a corpus of product reviews. My corpus contains 50,000 samples, of which I take 70% for training and 30% for testing. I discretized the 5-star rating to 3 ...
0
votes
1answer
2k views

Any way to handle plurals in NLP without stemming words [closed]

I am playing around with topic modeling, and I notice a lot of related plural and singular words in my corpus (e.g. 'champion' and 'champions' are both found). I gave the porter stemmer in NLTK a ...
2
votes
2answers
510 views

Tagging of tweets using NLTK

Is there a method to perform tagging of tweets using NLTK? The pos_tag() function gives incorrect results on twitter data (which uses textese): ...
2
votes
2answers
1k views

Is there any package in R/Python which can analyze Pos./Neg sentiment of whole review? [closed]

I'm a newbie here in the forum and new to text analytics using Python and R. My question is somewhat similar to Is there a better approach than counting positive-negative words in sentiment analysis? ...
6
votes
1answer
3k views

Difference between Log Entropy Model and TF-IDF Model?

I would like to understand what are the differences/advantages in using TF-IDF or the Log Entropy model for represeting documents and queries in an information retrieval system using diferent weights. ...
0
votes
1answer
389 views

k means clustering for larger text fields

I'm a beginner in data science/machine learning and am attempting to work through some problems on my own I am running a K-means clustering on a dataset consisting of "mission statements". These can ...
1
vote
1answer
356 views

How may I convert Perplexity to F Measure

In the practice of Machine Learning accuracy of some models are determined by perplexity, (like LDA), while many of them (Naive Bayes, HMM,etc..) by F Measure. I like to evaluate all the models with ...
1
vote
1answer
737 views

NLTK: odd outputs from bleu_score

For machine translation purposes I use bleu score, which seems to be the validation mechanism of choice (used in the sutskever 2014 sequence-to-sequence). The purpose is to get as high bleu as ...
2
votes
0answers
448 views

Cross Co-occurrence between two corpora

I've looked around for a solution to this problem specifically in nltk, quite a bit but couldn't find much help either on SO or elsewhere. My problem is as follows: I have a set of aligned pairs of ...
0
votes
1answer
242 views

How to find similar document with new vocabulary

I am working on a problem of finding similar documents. I am using a Tf-Idf based vector space model representation of documents and it gives me good results. However when I encounter a document ...