0
votes
0answers
8 views

automatic assign class name based on text

My question is , I have a set of plain text , i want to create category based on the text. Eg: i have written something about Soup recepie then the algorithm must create a category called Food. After ...
1
vote
1answer
34 views

Assigning meaningful cluster name automatically

The objective of my work is to cluster the text documents. Once the documents are clustered, traditionally the system will assign numeric value for the clustered group. For example if I have 5 ...
2
votes
1answer
50 views

Selecting number of clustering classes automatically

I am working in text clustering. I would like to find a way to identify the number of classes for the clustering process automatically rather than proving the number of class manually. Is their any ...
0
votes
1answer
66 views

Clustering structured data: Assessing the similarity of documents that appear in tree structure

Usually when performing text document clustering, similarities across documents are assessed based on the lexical content of documents. But, in my problem, I wish to consider both the lexical content ...
2
votes
1answer
69 views

How can I evaluate the performance of a system that generates word clusters?

The word2vec tool uses deep learning to compute vector representations of words. They've mentioned that - "The word vectors can be also used for deriving word classes from huge data sets. This is ...
0
votes
1answer
40 views

Data Conversion to Standard data format in hierarchical Dirichlet process

I'm trying to test the performance of posterior inference on a set of documents with hierarchical Dirichlet process for topic modeling. How can i convert my data (document) to standard data format ...
2
votes
1answer
135 views

Which papers discuss classification or clustering of source code according to programming language?

My specific problem is to separate a huge archive of files containing source code and sometimes including embedded languages (apart from the main language).
1
vote
1answer
120 views

Clarification needed about min/sim hashing + LSH

I have a reasonable understanding of the technique to detect similar documents consisting in first computing their minhash signatures (from their shingles, or n-grams), and then use an LSH-based ...
3
votes
1answer
181 views

Keyword clustering

I have one million of keywords (from search queries in google), and I need to group them semantically. I have already done some research and I have found information about how to extract keywords and ...
1
vote
1answer
90 views

Text clustering papers with Precision and Recall

I have created a text clustering algorithm and calculated the Precision and Recall measures for the evaluation. I am looking for papers that contain other text clustering algorithms evaluations with ...
0
votes
2answers
91 views

Find matching/common questions within a set of surveys

I'm currently working on cleaning/preprocessing a bunch of survey data from a collection of similar but distinct surveys. In order to combine the survey results this involves, among other things, ...
0
votes
2answers
134 views

Tf-idf Without the Log Function

Bear with me: I have a data set that is begging for the tf-idf transformation to account for a long tailed distribution of degree. Right now, it is in the form of a network where a tie represents ...
0
votes
1answer
179 views

bag of words in an online configuration, for classification / clustering

I have a set of image documents. I extract text keywords from this images using OCR to represent each image as a bag of words (a vector where each value is the number of occurrence of a word in the ...
3
votes
1answer
2k views

Euclidean Distance b/t unit vectors or cosine similarity where vectors are document vectors

I was reading Similarity Measures and suddenly my whole world was falling apart. I have implemented a search engine using clustering techniques. For clustering, I used k means which uses Euclidean ...
5
votes
5answers
5k views

Is cosine similarity a classification or a clustering technique?

In document classification, is cosine similarity considered a classification or a clustering technique? But you need training data with the cosine similarity for creation of the centroid right?
1
vote
1answer
838 views

What are the differences between document classification and clustering when working with a single topic?

I am doing some web page clustering work and I'm going to use cosine similarity as my distance measure. Even though cosine similarity is a clustering technique, I have to give training data in order ...
5
votes
1answer
3k views

The input parameters for using latent Dirichlet allocation

When using topic modeling (Latent Dirichlet Allocation), the number of topics is an input parameter that the user need to specify. Looks to me that we should also provide a collection of candidate ...
5
votes
2answers
939 views

Algorithms for clustering documents by similar words and phrases

I'm working on a project where I'm trying to take a pair of documents and find and group (cluster) similar words and phrases between them. Which algorithm would solve this kind of a problem? I know ...
4
votes
2answers
1k views

How to plot results from text mining (e.g. classification or clustering)?

In text classification and clustering, the number of features are normally big, e.g. I currently get are around 5,000 features which is already really small compared to many other text mining tasks. ...
4
votes
0answers
160 views

Category selection for text classification

It is said that to achieve a good result (many different metrics) for text classification, it is not always a business of selecting the algorithm/classifier. Sometimes, it is even more important to ...
2
votes
0answers
88 views

Statistical analysis on categories before text classification

I want to classify text by different topics. However, one of the current problems is that there are several topics/categories that are quite intuitively independent and statistically standalone, but ...
3
votes
2answers
473 views

Clustering of 10's of millions of high dimensional data

I have a set of 50 million text snippets and I would like to create some clusters out of them. The dimensionality might be somewhere between 60k-100k. The average text snippet length would be 16 ...
0
votes
0answers
138 views

Understanding text Clustering

I have not had much luck explaining my question so please do make a comment if you don't understand what I am asking. Core Question: How can I approach the problem of clustering text[not text ...
5
votes
3answers
326 views

Semantic distance between excerpts of text

I'm wondering how far along the natural language processing is in determining the semantic distance between two excerpts of text. For instance, consider the following phrases Early today, I got up ...