Statistical analysis of texts expressed in languages spoken or written by people, such as English or Norwegian.

learn more… | top users | synonyms

1
vote
1answer
24 views

Capturing initial patterns when using truncated backpropagation through time (RNN/LSTM)

Say that I use an RNN/LSTM to do sentiment analysis, which is a many-to-one approach (see this blog). The network is trained through a truncated backpropagation through time (BPTT), where the network ...
1
vote
0answers
17 views

No need for standardization with Adadelta? (RNN/LSTM)

Often it is best to standardize data before inputting it into a machine learning algorithm. This is also the case with deep learning algorithms such as convolutional neural network. However, when ...
2
votes
1answer
61 views

Word2vec representation used for RNNs turn a word into a scalar or a vector?

Based on this post on quora.com and other sources I got the impression that each word in the word2vec representation is represented by a vector containing e.g. 500 dimensions. However, when looking ...
0
votes
0answers
17 views

Incorporating letter transition model into linear-chain CRF

Suppose I have a linear-chain CRF for e.g. handwriting recognition, $$ p(\mathbf{y}\mid\mathbf{X}) = \frac{1}{Z_\mathbf{X}}\exp\left(\sum_{j=1}^m\mathbf{w}_{y_j}^T\mathbf{x}_j + ...
0
votes
0answers
20 views

Calculating gradient in a neural probabilistic language model

I am trying to reproduce a neural probabilistic language model described by Bengia (without distributed softmax computation). According to Bengio et al. "training is achieved by looking for model ...
2
votes
1answer
36 views

Benefits of clustering algorithms and Latent Dirichlet Allocation / topic models for finding clusters of words / topics in text

I am interested in finding clusters of words / topics in text. I am trying to learn more about potential approaches. The Wikipedia page on document clustering seems to provide a helpful overview ...
3
votes
2answers
36 views

Correct and clear wording for non-causal correlation

Despite reading multiple statistics and epidemiology texts as well as studies, I have trouble describing the following in plain English for a public of doctors (so, non-statisticians or biomedical ...
0
votes
0answers
31 views

“without” not in NLTK Stopwords corpus, while “with” is

I will preface this by saying that I do not have experience in linguistics. Unnecessary Background: I was doing some NLP in python using NLTK, and began by removing stopwords from my text of ...
0
votes
0answers
13 views

Varying transition probabilities by position

I'm still very new to Bayesian Tables, Hidden Markov Models and the likes, but have an otherwise solid computational and linguistics background. I've been diving into NLTK (Natural Language Toolkit) ...
3
votes
1answer
69 views

Understanding the use of logarithms in the TF-IDF logarithm

I was reading: https://en.wikipedia.org/wiki/Tf%E2%80%93idf#Definition But I cannot seem to understand exactly why the formula was constructed the way it is. What I do Understand: iDF should at ...
1
vote
1answer
57 views

Is this interpretation of sparsity accurate?

According the documentation of the removeSparseTerms function from the tm package, this is what sparsity entails: ...
2
votes
0answers
45 views

How is prior knowledge of letter/word patterns incorporated into handwriting (or speech) recognition?

Using handwriting recognition as an example, we can train various models to recognise individual characters but to actually be useful we must incorporate prior knowledge of common character sequences, ...
2
votes
1answer
54 views

Estimating the best length of n-gram

I have a long sequence of words or letters {word1 word2 word3 word1 word1 word2 ..etc}. Lets say we extract all the ngrams (unigrams, bigrams, trigrams, 4-gram, 5-gram ....) along with their frequency ...
0
votes
0answers
22 views

Forming relational graph for a noun by mining web

I want to find a relational/relevance graph for any noun, by mining the web. For example the graph of sushi may be like : sushi -> fish(seafood),rice-> Japanese -> Food. PS : I may be missing some ...
2
votes
2answers
69 views

How can I improve feature selection for my Naive Bayes Classifier?

I am classifying companies into two classes ( a particular business type, or not that business type ), using a Naive Bayes Classifier. Specifically, I'm using PHP and PHP NLP Tools. I have two ...
2
votes
0answers
24 views

Natural Language measure of obfuscation

I have some experience with sentiment analysis in natural language processing, but want to learn some new algorithms and techniques for a project I am working on. In particular, I am interested in a ...
1
vote
0answers
20 views

Different Methods for clustering skills in text

Consider a talent pool in which each member has some set of skills. Some of these talent are submitted to orders as potential candidates of which one is selected. It is reasonable to assume that the ...
4
votes
3answers
148 views

Is it feasible to use k-Nearest Neighbours to identify text language?

I have seen various language identification libraries that claim to use naive Bayes classifier for text language identification, like CLD2 and language detector, but not any library that uses other ...
0
votes
0answers
12 views

What are good tutorial on Weighted Finite Automata?

I would especially appreciate papers, books or tutorials with source code already available. Currently I'm reading "Spectral Learning Techniques for Weighted Automata, Transducers, and Grammars" by ...
0
votes
0answers
52 views
1
vote
0answers
39 views

What machine learning algorithm should I choose to fill in blanks from context?

I have a project where I need to be able to fill in a missing word given a few words of context. In other words, suppose I have a sentence: I went ____ the store. I want to be able to deduce ...
3
votes
1answer
741 views

Why is n-gram used in text language identification instead of words?

In two popular language identification libraries, Compact Language Detector 2 for C++ and language detector for java, both of them used (character based) n-grams to extract text features. Why is a ...
0
votes
0answers
19 views

References for learning text stemming

I am trying to learn and experiment with text stemming. My ultimate goal is knowledge extraction from scientific text and corpus with emphasis on contextually multiplicity. But text stemming and ...
1
vote
1answer
50 views

How to get corpus used for word-frequencies for languages other than english?

I have taken a corpus of English for finding the word frequency in giving recommendation for spelling mistakes. I just used simple : Edit Distance between ...
6
votes
1answer
51 views

Has there been a project to apply machine learning to generation of indices for books?

Generating an index for a textbook is a tedious task. Can one automate it with machine learning? Are there any references to previous attempts in the literature to do this?
0
votes
0answers
25 views

What statistical/statistical analysis can I use to extract information from set of words?

I have long (thousands) lists of words set. A set of words is usually combination of a number and unit of measurement, but it can also be a combination of words. What kind of analysis can I use to ...
4
votes
1answer
70 views

How to make an effective sampling from a database of text documents?

Problem: I want to know methods to perform an effective sampling from a database. The size of the database is about 250K text documents and in this case each text ...
0
votes
1answer
82 views

Question about Continuous Bag of Words

I'm having trouble understanding this sentence: ...
2
votes
1answer
155 views

How to handle changing input vector length with neural networks

I want to train a neural network with a sequence of character as an input vector. Learning examples have different length and for this reason I don't know how to represent them. Let's say I have two ...
0
votes
0answers
107 views

Sentiment Analysis - How should I handle negatively biased word list length?

I'm implementing a simple sentiment analysis algorithm where the authors of the paper have a word list for positive and negative words and simply count the number of occurrences of each in the ...
1
vote
0answers
61 views

How to understand the patterns of section names in a resume?

recently I am doing some text mining works with resumes. The objective is to divide the resume into several sections based on its headings and contents and then classify it for required jds. Eg. We ...
0
votes
1answer
42 views

Word-frequency and statistics

I am new to statistics and am wondering how I can apply it in linguistics. There is a conjunction in a corpus that 902 times (.91) conjoins sub-clausal units, and 91 (.09) times clausal units. How ...
0
votes
0answers
23 views

Using LIWC for ANOVA and pairwise analyses

I want to use a tool called Linguistic Inquiry and Word Count (LIWC) to analyze text data. As far as I know, it counts the number of words found in a given text that match a set of dimensions (e.g., ...
2
votes
1answer
34 views

Comparing corpus complexities

I would like to compare how complex (varied or predictable) are my three corpora. They are from different topics, so some vocabulary is different, some is the same. Looking at one of the data sets ...
0
votes
0answers
18 views

How alignment between words are generated by IBM model1?

I implemented the translation model IBM1. As a result, I got the translation table P(targetWord|sourceWord) wich is ok. I want also to obtain the alignment of words in the corpus that I used in order ...
2
votes
3answers
64 views

A good intro to computational linguistics?

I have a pretty good background in data analysis and statistics in the social sciences, including both frequentist and Bayesian paradigms, and I have recently been introduced to computational ...
1
vote
0answers
25 views

How to learn (arbitrary) constraints for selecting the best candidate from a group?

In my classification problem, each instance is a group of possibly hundreds of candidates, from which only one should receive the label $True$ and the remainder the label $False$. For example, in ...
0
votes
2answers
397 views

How to standardize text data for training Neural Networks?

I want to train neural network with text data(natural language) as input for classification purpose. One way for standardizing text data for neural network is to use N-GRAM/SKIP-GRAM representation ...
1
vote
1answer
128 views

How to find the perplexity of a corpus

The formula of the perplexity measure is: $$p: \left(\frac{1}{\sqrt[n]{p(w_1^n)}}\right)$$ where: $p(w_1^n)$ is: $\prod_{i=1}^n p(w_i)$. If I understand it correctly, this means that I could ...
2
votes
1answer
28 views

determining significance of term use

Thing one: feel free to RTFM me: I'm definitely looking for search-able terms or background reading. Our situation is this: we have a set of 140 reviewers and 20 elements. Each reviewer reviews each ...
1
vote
1answer
64 views

Neural language model training - stochastic vs batch

Dealing with a very basic neural language model: 3 words of context, vector size 100, one hidden layer size 200, vocabulary size 1000, predicting the next word with a softmax output layer. Previously ...
2
votes
0answers
19 views

How to measure how 'well' I am matching Google keywords?

For google keywords you can bid on a broad match. For example let's say I bid on the keyword 'best hamburger' and somebody searches 'What sort of beef makes the best hamburger?' and 'eat best ...
0
votes
0answers
48 views

Hashing functions in NLP

I have been reading a lot of papers about nlp which use the hashing trick, and I came across a lot of sentences like : "We take k hashing functions to hash words or bi-grams". And after that they ...
0
votes
1answer
120 views

How do search engines generate related searches?

I would like to know how search engines like Bing generate related searches when the user starts typing into the search box. From what I gather, there has to be some sort of a ranking algorithm where ...
0
votes
1answer
51 views

How can I show that my sample is random and a good representation of the population?

Okay, I'm looking at a population of user reviews. I have collected a random sample of the reviews and studied the trends of the words used and sentence structure. How do I make inferences about ...
0
votes
0answers
73 views

Trying to understand the basics of a mixed-effects logistic regression model for a 10-step continuum

I am trying understand how to correctly build a mixed-effects logistic regression model in R. I believe my model is pretty simple and straight forward but I'm lacking in experience and uncertain I'm ...
0
votes
0answers
192 views

Any advice on how to improve my accuracy rate in text classification?

I'm trying to do a text classification task. Here are some specs: Context file size = 1M+ documents already labeled Number of top-labels = 17 Number of sub-labels = around 130 Each document is ...
0
votes
0answers
35 views

Unsupervised feature learning from raw text as a previous step for clasification?

I have a corpus of 2500 opinions, is it posible to use scikit´s restricted boltzmann machine implementation to extract a feature vector as a previous step to a classification task?. What aproach do i ...
2
votes
2answers
88 views

Weighting words based on position in text

I'm currently working on semantic analysis and had a question about text organization and structure. Are there any algorithms, or statistical / machine-learning models that weight the importance of a ...
1
vote
1answer
200 views

How to handle unseen features in a Naive Bayes classifier?

I am writing a naive bayes classifier for a text classification problem. I have a bunch of words and an associated label: ...