Natural Language Processing is a set of techniques from linguistics, artificial intelligence, machine learning and statistics that aim at processing and understanding human languages.

learn more… | top users | synonyms (1)

0
votes
0answers
5 views

Transition/emission probabilities POS tagger English

is there an freely available online version of transition and emission probabilities for an HMM model used for POS tagging English text? It seems like there are many powerful existing taggers out ...
1
vote
0answers
27 views

Pros and Cons: LDA vs Neural Networks

LDA is an older approach for word representations, there are newer methods now like CBOW and Skip-gram. But what are the improvements of these models? Do they improve in every way or does LDA still ...
0
votes
0answers
23 views

How to detect the events using NLP and Machine learning?

I have text describing about events such as birth , new job , wedding , death etc .. or no event . How do i detect these events ? My approach is to form set of words and search them in text ...
0
votes
0answers
17 views

How large should be negative train set in text classification of rare category

I'm working on the classification of medical texts in order to find texts about the quite rare disease from the big set of all medical articles. I have the set of positive examples and some negative ...
0
votes
0answers
51 views

Kneser Ney smoothing, why the maths allows division by 0?

I'm studying Natural Language Processing and the various smoothing approaches. I'm finding a little hard to understand how to handle unknown words with the Kneser-Ney smoothing. In particular I'm ...
0
votes
0answers
18 views

Evaluating kmeans clustering with silhouette coefficient, weird results

I'm performing a kmeans clustering on a 22.000 documents datasets. Not knowing how many clusters I should get, I ran different k values and try to assess the validity of the clusters by determining ...
0
votes
0answers
11 views

How to choose an optimal dimension reduction factor in LSA processing

I'm performing a K-Means clustering on a 400.000 text dataset. After eliminating useless chars and removing stopwords, I get a dictionnary size of around 42000 words. So before doing the clustering, ...
2
votes
0answers
20 views

Overview research on ambiguity of words

As far as I know vector representations of word embeddings do not account for ambiguity. A single word can have different meanings f.e. "hot" can mean very good looking or very warm. Another example ...
4
votes
1answer
41 views

Do word vectors obtained via word embedding techniques really form a vector space?

Word embedding refers to feature learning techniques in natural language processing where words are mapped to vectors of real numbers in a low-dimensional space, the embedding space. Similar ...
1
vote
1answer
90 views

What is the perplexity of a mini-language of numbers [0-9] where 0 has prob 10 times the other numbers?

I'm reading Speech and Language Processing, Jurafsky and Martin, in particular chapter 4 where they introduce perplexity see https://web.stanford.edu/~jurafsky/slp3/4.pdf (page 8-9) Here a brief ...
0
votes
0answers
16 views

Create features from a document

I have been given an assignment related to NLP and I am a newbie in this field. Train a named entity recognition system that treats the documents as strings of mentions, x . A labelling of the ...
0
votes
0answers
8 views

trainset and testset with different distributions

I am looking for the research/papers that shows that more balanced train leads to a better macro performance on highly unbalanced test. For example, there is multi-label problem, where 80% of the ...
0
votes
1answer
15 views

How may I convert Perplexity to F Measure

In the practice of Machine Learning accuracy of some models are determined by perplexity, (like LDA), while many of them (Naive Bayes, HMM,etc..) by F Measure. I like to evaluate all the models with ...
0
votes
0answers
16 views

Is there any advantage of using MEMM instead of CRF for named-entity recognition?

I wonder whether there is any advantage of using maximum-entropy Markov model (MEMM), a.k.a. conditional Markov model (CMM) instead of using conditional random fields (CRF) for named-entity ...
1
vote
0answers
40 views

Katz Backoff help calculating alpha

$Pkatz(z|x,y) =$ $P'(z|x,y), if C(x,y,z) > 0$ $α(x,y)Pkatz(z|y), else if C(x,y) > 0$ $P'(z), otherwise.$ $Pkatz(z|y) =$ $P'(z|y),ifC(y,z)>0$ $α(y)P' (z), otherwise.$ ...
0
votes
0answers
17 views

Alpha on Katz Backoff using Simple Good-Turing

I'm building an n-gram language model to predict the next word, I've implemented a simple Good-Turing smoothing on all my probabilities and have calculated the P0(mass probability of unseen event). I ...
1
vote
0answers
16 views

Is there any package in R for Sound like analysys of text [closed]

The words "Jhon" and "Joan" may sound similar although spelling is different. Is there any package for "Sound like Analysis" in R
0
votes
1answer
25 views

Doubt about feature selection

I'm working on a text classification problem using Python and NLTK. I've got two frequency distributions, one for each class (it's basically a binary classification). So, my doubt it's if there's a ...
1
vote
0answers
25 views

Simple Good-Turing Probabilities higher than old probabilities

I've implemented Simple Good-Turing to get new probabilities of unigrams on my Corpus. Everything works fine. I'm just confused on how come the probability of a word after the Good-Turing discount be ...
0
votes
0answers
20 views

nltk: odd outputs from bleu_score

For machine translation purposes I use bleu score, which seems to be the validation mechanism of choice (used in the sutskever 2014 sequence-to-sequence). The purpose is to get as high bleu as ...
1
vote
0answers
20 views

How to create a realistic multi-label data set of web pages? [closed]

I am working on a project which is a web filter. The web filter is supposed to classify web pages using any multi-label classification technique and then block them if anyone of the category is found ...
1
vote
1answer
54 views

Handling unknown words in language modeling tasks using LSTM

For a natural language processing (NLP) task one often uses word2vec vectors as an embedding for the words. However, there may be many unknown words that are not captured by the word2vec vectors ...
1
vote
0answers
150 views

Text classification with word2vec and neural nets [spacy.io, keras]

I have about 300.000 messages with bodies and titles at hand. ~20% are of them labeled positive. Right now, I run the word2vec feature generation with spacy.io (excellent library btw.), generating ...
0
votes
0answers
9 views

Stacked CRF implementation

I am looking for a library that can train a stacked conditional random field (CRF). I plan to use it for natural language processing purposes. Ideally, Python interface, works on Linux, and ...
0
votes
0answers
14 views

Neural Network Structure Sentence

I'm new in Stats SE. I'm trying to figure how can I can give a preprocessed sentence (with dependency parsing structure and pos tags), and prepare a training set, to my network be able to predict the ...
0
votes
0answers
17 views

How do you learn GLove word vectors?

$X_{ik} = \# $ times work $i$ occurs in the context of work $k$. $X_i$ is the number of times word $i$ occurs. $P_{ik} = \frac{X_{ik}}{X_i}$. It's not clear to me what equation 3 solves and why the ...
3
votes
0answers
33 views

The Effect of Stopword Filtering prior to Word Embedding Training

Recently I have played with the pretrained GLOVE word embedding model for Twitter http://nlp.stanford.edu/projects/glove/ I notice that common stopwords are existing in the model. That is, there is ...
0
votes
0answers
19 views

Does similarity with word2vec or LDA include “is_a” similarity?

Various methods including word2vec and latent dirichlet allocation (LDA) extract similarities between words in terms of (bit different) similarity criteria. On LDA, "baseball" and "football" is ...
0
votes
0answers
10 views

Gaming the ROUGE metric for text summarization

ROUGE seems to be the standard way of evaluating the quality of machine generated summaries of text documents by comparing them with reference summaries (human generated). $$ROUGE_{n}= \frac ...
0
votes
1answer
32 views

Why using Autoencoder in nlp seem hard to do?

I want to implement an autoencoder with theano to express each article. When I made it following the tutorial of UFLDL. I found that it's hard to get gradient descent(I use LBFGS). I want to know ...
0
votes
2answers
84 views

How to implement question answering program based on Q&A data?

For example I have QA data: Q: Do you like apples? A: Yes. Q: Do you like running? A: Yes. Algorithm should take that input, thesaurus(synonyms, antonyms, etc.), word categorization ...
0
votes
0answers
28 views

How may I work out Name Entity Resolution?

I am trying to work out a Name Entity Recognition (NER) problem. I am presently trying to work around two supervised approaches of Maximum Entropy (MaxEnt) and (HMM). I like to extend the work to Name ...
0
votes
0answers
75 views

Similarity of learning in doc2vec and word2vec

I have understood word2vec algorithm from Richard Socher's Lecture notes. Basically he uses one input matrix with one-hot vectors of words from vocabulary ...
0
votes
0answers
14 views

What is the best method to extract data from speech recognition data?

For my project i need to extract the action details from the voice command given. A voice command may consist of unnecessary data such as "as the tile","at the end" etc. What is the best way to ...
0
votes
0answers
10 views

NLP - Question Answering

I would like to make a toy example to simple Question Answering algorithm. I have a small e-commerce selling beer, and i would like to answer simple queries like: "I would like a light and cheap beer" ...
0
votes
1answer
28 views

How to estimate the number of samples needed for NLP corpus?

What are approaches to estimate how many samples are needed for an SVM classification task in features extracted from text? If I have for example a set of 8 labels in a 100-dimensional featurespace. ...
1
vote
1answer
27 views

CRF implementation with Python API that allows a CRF model to be trained multithreadedly

I am looking for a conditional random field (CRF) implementation with a Python API that allows a CRF model to be trained multithreadedly. I currently use pyCRFsuite, which works great except that CRF ...
1
vote
0answers
16 views

Where is the network in memory networks?

The paper Weston, Jason, Sumit Chopra, and Antoine Bordes. "Memory networks." arXiv preprint arXiv:1410.3916 (2014). introduces memory networks. The given definition is as follows: Where is the ...
3
votes
1answer
29 views

Sequence length when training a conditional random field (CRF)

I am training a conditional random field (CRF) to perform named entity recognition. I have 1000 documents, each containing from 100 to 500 sentences. During the training phase, is it better to train ...
0
votes
0answers
23 views

How does TextRank differentiate between keywords and simply frequent words, such as “is”

I am trying to understand how TextRank document summary algorithm works. A few articles that I've read so far introduce text rank as a modification of page rank (e.g. article in wikipedia). However, ...
1
vote
0answers
41 views

How large corpus is necessary for word2vec?

I'd like to train word2vec model with blog posts I collected via a search engine. I've collected around 1000 entries. Do you think I can get a valid model by letting word2vec learn such small corpus? ...
0
votes
0answers
7 views

NLP POS Tagger For A New Domain

I am using NLTK and would like to use an existing pos-tagger which has been pre-trained and further train it for a new domain. What I understand from Perceptron tagger operation is that it looks up ...
2
votes
1answer
34 views

How may I create a large annotated Corpus for training?

I am trying to create an annotated corpus of few million words. I want to use it as training data for some supervised algorithm. I may try to implement a task like Parts of Speech(PoS) Tagging or Name ...
0
votes
1answer
97 views

Difference between dynamic pooling and static pooling in convolutional neural networks

Since yesterday I was thinking that pooling layer in CNN has fixed size(e.g. 2 by 2). Then I saw in this paper: http://phd.nal.co/papers/Kalchbrenner_DCNN_ACL14 We define a convolutional neural ...
0
votes
0answers
23 views

Calculate statistical significance in natural language processing

I have a task to say whether the difference in performance between two systems is statistically significant. The task is similar to sentiment analysis. I have sentences and I need to classify them ...
0
votes
0answers
17 views

Latent Dirichlet allocation: how to derive $\theta$ given $\alpha$?

I have been studying Latent Dirichlet allocation (LDA) since quite long. I have a confusion in Dirichlet priors. For example, if I consider 3 topics and take $\alpha_1 = 0.1$, $\alpha_2 = 0.1$ and ...
0
votes
0answers
58 views

Build HMM of text data in R

I'm trying to make my own HMM tagging in R but don't know how to estimate parameter values since the packages I have been working with haven't worked with my data. The latest package I have been ...
2
votes
1answer
218 views

How does Word2Vec's skip-gram model generate the output vectors?

I am having problems understanding the skip-gram model of the Word2Vec algorithm. In continuous bag-of-words is easy to see how the context words can "fit" in the Neural Network, since you basically ...
1
vote
0answers
95 views

calculating PMI for co-occurrences of words

I am in the process of building a question answering system. I am interested in calculating the PMI for words $x$ and $y$ occurring within 5 words of each other in a document. I have the formula and ...
3
votes
1answer
153 views

In word2vec, for analogies do we use “in” or “out” vectors?

In word2vec each word is associated with two vectors (one for in and one for out) so that it predicts conditional probability: $$P(word_{out}|word_{in}) = \frac{\exp(v_{in} \cdot ...