0
votes
1answer
65 views

Using topic words generated by LDA to represent a document

I want to do document classification by representing each document as a set of features. I know that there are many ways: BOW, TFIDF, ... I want to use Latent Dirichlet Allocation (LDA) to extract ...
1
vote
1answer
164 views

Feature selection : how to select the Information Gain threshold?

I am trying to use Information Gain to select features when classifying text with a Support Vector Machine. For each word in our training data, we computed its information gain. Then, we should keep ...
2
votes
2answers
175 views

Selecting a feature modeling approach for text classification

I am new to text processing. Currently I am trying to determine which type of feature vector I need for a classification problem. I am mainly deciding between binary feature modeling and ...
2
votes
2answers
126 views

improve precision in text classification

I am working on binary text classification using sklearn: The length of each sample is not high (~ 200-500 characters) I use TF-IDF to get important words as TfidfVectorizer(sublinear_tf=False, ...
1
vote
0answers
76 views

How to explain difference of importance between feature selection and model quality?

I have a data collection with a mixed feature set consisting of both numerical features and text features. The number of numerical features is quite small, i.e., 6, comparing to the number of text ...
1
vote
1answer
813 views

How does scikit-learn perform $\chi^2$ feature selection on non-categorical features?

I'm experimenting with $\chi^2$ feature selection for some text classification tasks. I understand that $\chi^2$ test checks the dependencies B/T two categorical variables, so if we perform $\chi^2$ ...
3
votes
2answers
296 views

How to do feature selection for learning from positive and unlabeled examples?

I have a binary classification task for German webpages for which I only have positive examples. That is why I use learning from positive and unlabeled examples as described on this page, also known ...
0
votes
1answer
107 views

Lucene-based text feature construction

When doing the feature construction for text mining, does Lucene has a better performance in terms of classification/clustering result than the traditional bag-of-word approach?
2
votes
0answers
116 views

Feature construction for text mining

In the text mining, besides N-gram model, what are the state-of-art models for building feature space while capturing the dependence among the different words, or capturing the semantic meaning in the ...
3
votes
1answer
923 views

Feature selection methods for document classtification

I have a simple document classification problem where i need to classify some documents to a definite set of classes. I need to perform a feature selection (where I will select the most important ...
4
votes
2answers
844 views

Feature selection for the text mining?

Before performing the task of text mining, we need to select the features for characterizing each given document. Are there any systematic guidance on choosing the document features? How does the ...
0
votes
1answer
181 views

How to get scored combination of features

My data looks like this (F=Features) ...