The tag has no wiki summary.

learn more… | top users | synonyms

0
votes
0answers
9 views

Unsupervised feature learning from raw text as a previous step for clasification?

I have a corpus of 2500 opinions, is it posible to use scikit´s restricted boltzmann machine implementation to extract a feature vector as a previous step to a classification task?. What aproach do i ...
2
votes
2answers
78 views

Weighting words based on position in text

I'm currently working on semantic analysis and had a question about text organization and structure. Are there any algorithms, or statistical / machine-learning models that weight the importance of a ...
1
vote
1answer
49 views

How to handle unseen features in a Naive Bayes classifier?

I am writing a naive bayes classifier for a text classification problem. I have a bunch of words and an associated label: ...
0
votes
0answers
17 views

Feature learning with a deep learning aproach?

How to create a feature vector from text with a deep learning aproach?. Im new at this topic, could anybody advice me where to start and how to aproach this task?.
2
votes
0answers
38 views

Understanding Singular Value Decomposition in the context of LSI

My question is generally on Singular Value Decomposition (SVD), and particularly on Latent Semantic Indexing (LSI). Say, I have $ A_{word \times document} $ that contains frequencies of 5 words for ...
0
votes
0answers
60 views

What algorithms should I use to perform job classification based on resume data?

Note that I am doing everything in R. The problem goes as follow: Basically, I have a list of resumes (CVs). Some candidates will have work experience before and some don't. The goal here is to: ...
1
vote
1answer
46 views

Calculating Perplexity

In the Coursera NLP course , Dan Jurafsky calculates the following perplexity: Operator(1 in 4) Sales(1 in 4) Technical Support(1 in 4) 30,000 names(1 in 120,000 each) He says the Perplexity is 53. ...
2
votes
1answer
26 views

Question about the probability chain rule

I've understood from this: Is this a correct statement of the probability chain rule? that in the chain rule for probability, conditioning can be done on different variables. I was wondering what ...
1
vote
1answer
134 views

Naive Bayes with unbalanced classes

As a part of a project for the university is should train a Naive Bayes classifier to classify question and answers in three different categories, the task should be easy since that the 3 classes are ...
1
vote
0answers
81 views

Visualizing Mutual Information Against TF-IDF for Text Corpus Data

I'm working on a data visualization project for the semester and have decided to work with a corpus of discussion forum data focused around debate over political issues (available here). I'm ...
1
vote
1answer
66 views

Hidden Markov Model: Predict observation sequence from state sequence

Given a transition matrix, starting probability, means and covariances Is it possible to predict the most likely observed sequence for a given state sequence using the above details? If yes, how? ...
0
votes
0answers
28 views

Issue With class imbalances in classification problem

I am working on a word sense disambiguation problem. Specifically, I am using decision lists to classify the ambiguous word. Decision lists work in the following way. The quantity ...
0
votes
0answers
32 views

Word Sense Disambiguation in Practice

I have a question that might seem very obvious but I don't really have a good answer for it. There are many algorithms out there that deal with word sense disambiguation but all of the ones that I ...
1
vote
0answers
50 views

How to extract structured information from a text string?

I have a text string containing unstructured data and I would like to analyze it in order to extract structured information. In particular, this text string specifies when a service is operational ...
1
vote
0answers
72 views

Keyword probabilities

I'm not a huge stats buff and am wondering what the best approach to my problem is. Say I have a list of PPC* keywords and if a desired action was taken. Let's say I put this data into two word ...
0
votes
0answers
143 views

MEMM label bias problem

I study MEMM model with application to POS tagging. The question I found that the MEMM model has one drawback - "label bias problem". More formally, The transition leaving a given state compete ...
0
votes
1answer
77 views

Multiple Bernoulli and Multinomial Distirbution

It's well known that language can be modeled by Multinomial distribution and Multiple Bernoulli distribution. So far I don't see any advantage of Multiple Bernoulli distribution representation over ...
2
votes
0answers
49 views

Expectation Maximization Clarification

I found very helpful tutorial regarding EM algorithm. The example and the picture from the tutorial is simply brilliant. Related question about calculating probabilities how does expectation ...
0
votes
1answer
37 views

Improvement on duplicating instances

I have a task of Relationship extraction. There are some set of predefined relations in the corpus. I need to train classifier to recognize the type of relation or the lack of relation between every ...
1
vote
2answers
125 views

Most important journals in data mining/ML, NLP and IR?

Can you please provide with me with the names of the most important journals in data mining, machine learning, natural language processing and information retrieval?
1
vote
0answers
58 views

Calculating and Normalizing ngram relevancy scores from free text extraction

I currently look for a set of ngrams in many sets of documents to establish a relevancy score for each set - eg. I look for the n-gram "adhesive tape" in ~1M sets of 1-500 documents. The values I ...
0
votes
0answers
43 views

How to build a relevant text classifier?

I would like to build a message classification system which classifies a given message into either of 2 class - Relevant/Not. I don't have any labelled dataset. I only have certain keywords which ...
0
votes
0answers
217 views

Katz backoff for n-gram language models

I have defined unigram, bigram and trigram (language) models with my training data. Now I am checking their fitness towards a test data by using Katz backoff with a fixed discount of 0.5 (mo ...
0
votes
0answers
22 views

Evaluate text rarity in document set

I would like to evaluate the rarity of each sentence in a document set. Please let me know the state-of-the-art or a survey paper on this task. I've already checked several papers in the document ...
2
votes
1answer
132 views

Regression analysis in R using text field?

I'm working in R. I'd like to run a regression analysis for predicting price against terms in a text field. I have a dataset of jewellery auction listings, with price paid, date, and an unstructured ...
0
votes
0answers
19 views

Typical sequence of analyzing text

While going about examining textual data, do you have any process flow that works for you? What I do as of now: data collection importing into a corpus coding cleaning preprocessing: whitespace, ...
1
vote
0answers
21 views

Early papers using statistical models for word completion?

I am trying to find references about (early, as in pre-2000, but i would really love something pre-1990) statistical language models in word completion (like T-9, or Google's search autocomplete), but ...
3
votes
2answers
145 views

Is LSA and topic clustering easier in European languages similar to English?

I was watching a talk on latent semantic analysis and the speaker described experience applying LSA and REALLY messy data. He concluded that it demonstrated the difficulty of disambiguation of ...
0
votes
1answer
63 views

Time-delayed neural networks - Learning the “max” layer

In Unified Architecture for NLP paper time-delayed neural network proposed as a way to deal with variable length input. Input window slides over sequence and label each output with "time", then next ...
5
votes
1answer
168 views

Using text mining/natural language processing tools for econometrics

I am not sure whether this question is fully appropriate here, if not, please delete. I am a grad student in economics. For a project which investigates issues in social insurances, I have access to ...
2
votes
1answer
97 views

Markov chain getting stuck due to insufficient data samples

There is a lot of theory on Markov models and output generation out there, but I cannot locate any information on models getting stuck. I'm trying to create a model of a data set using a Markov ...
7
votes
1answer
1k views

Intutive difference between hidden Markov models and conditional random fields

I understand that HMM are generative models, and CRF are discriminative models. I also understand how CRFs' are designed and used. What I do not understand is how they are different from HMMs'? I read ...
3
votes
0answers
121 views

Latent Semantic Analysis - Co-occurrence of words

Let $A[n\times m]$ represents the term-document matrix, where, $n$ is the number of terms and $m$ is the number of documents. This matrix can be composed into 3 matrices (SVD decomposition) such as, ...
0
votes
2answers
143 views

Understanding the derivation of an equation in LDA modeling

When reading the derivation of LDA models, I usually get the following equations. I do not quite understand the second step, where $p(\mathbf{z}_{-i},\mathbf{w}|\alpha,\beta)$ was removed. Is that ...
1
vote
1answer
711 views

Loop over Tokens in RapidMiner's Text Processing Plugin

is there any possibility to iterate over the tokens of a text document within RapidMiner? My first try was to window the document after tokenisation. But this seems very complicated. I'm doing this ...
2
votes
1answer
119 views

How to extract ngrams from ambigous text after lemmatization?

After lemmatization of text I have a sequence of sets of lemmas, because every word can correspond to more than one lemma. How should I extract ngram statistics based on that? The only thing that ...
1
vote
1answer
101 views

Independence assumption in maximum entropy models in NLP

I am reading Klein and Manning's notes on Maximum Entropy for Natural Language Processing. On slide 22, they have an equation saying, $P(C|D,\lambda) = \Pi _{(c,d)\in (C,D)} P(c|d,\lambda)$. I am not ...
3
votes
0answers
118 views

Comparing term-frequency distributions with unequal sample sizes?

Background I have several datasets of word frequencies where some datasets have much more data than others: from 3000 samples to 20000 samples. I also have large reference corpora with millions of ...
0
votes
1answer
190 views

Regarding the feature generation method with SVM-based classification method

When using SVM to build classifier for a collection of documents, we can use term occurrence, term frequency or even TF/IDF. I would like to know what are the main disadvantages of using term ...
2
votes
0answers
23 views

what is the way of dealing with textual valued feature vectors for classification task?

I aim to work on twitter data for sentiment analysis but I am curios for the way of dealing such a huge number of textual features (words). Is using the Bag-Of-Words approach is the best? However I've ...
1
vote
0answers
105 views

Using sentiment lexicons or all words processing for sentiment analysis?

I am learning sentiment analysis to apply it to twitter real time data to predict user's mood. I ponder about using which alternative way to do that data mining job. Use all words to process and ...
3
votes
1answer
340 views

Latent Dirichlet allocation Implementation

I'm looking for some LDA implementation. I know about this one, MALLET but it is coded in Java and I need some more performant. Can someone give me some reference?
5
votes
1answer
750 views

Why is tf-idf used in conjunction with SVMs for classifying documents?

Term frequency - inverse document frequency is term count within a document weighted against the term's ubiquity within the corpus. This weight is based on the principle that terms occurring in ...
2
votes
0answers
87 views

Using n-grams to find data that does not 'belong'

Recently I posted a question over in CS.SE dealing with methods of classifying data. Essentially the problem is that I have a collection of strings (100's of thousands). Most of these strings are ...
2
votes
1answer
64 views

How to update a dynamic language model dataset?

I'm a statistics novice and I need help with a natural language problem. I'm writing a word-prediction algorithm for a mobile app. I'm using a unigram language model of word/count pairs where count ...
9
votes
2answers
1k views

Topic models and word co-occurrence methods

Popular topic models like LDA usually cluster words that tend to co-occur together into the same topic (cluster). What is the main difference between such topic models, and other simple ...
4
votes
0answers
76 views

Language modeling: why is adding up to 1 so important?

(if this venue is inappropriate, feel free to migrate it) In many natural language processing applications such as spelling correction, machine translation and speech recognition, we use language ...
0
votes
1answer
104 views

Lucene-based text feature construction

When doing the feature construction for text mining, does Lucene has a better performance in terms of classification/clustering result than the traditional bag-of-word approach?
2
votes
1answer
463 views

Different size of vocabulary made by Weka and R's tm

I own around 40,000 text files for preprocessing (in purpose of document classification). I used R (with tm package) for prototype and now looking for a equivalent Java library for products. ...
1
vote
4answers
1k views

Software or libraries to create doc-term matrix

does anyone know some Java libraries to create the document-term matrix for a large number (50,000) of documents ? I wish this library encompasses preprocessing functionalities, like stop-word and ...