Tagged Questions

0
votes
1answer
92 views

bag of words in an online configuration, for classification / clustering

I have a set of image documents. I extract text keywords from this images using OCR to represent each image as a bag of words (a vector where each value is the number of occurrence of a word in the ...
2
votes
1answer
613 views

Euclidean Distance b/t unit vectors or cosine similarity where vectors are document vectors

I was reading Similarity Measures and suddenly my whole world was falling apart. I have implemented a search engine using clustering techniques. For clustering, I used k means which uses Euclidean ...
4
votes
5answers
3k views

Is cosine similarity a classification or a clustering technique?

In document classification, is cosine similarity considered a classification or a clustering technique? But you need training data with the cosine similarity for creation of the centroid right?
1
vote
1answer
389 views

What are the differences between document classification and clustering when working with a single topic?

I am doing some web page clustering work and I'm going to use cosine similarity as my distance measure. Even though cosine similarity is a clustering technique, I have to give training data in order ...
3
votes
1answer
489 views

The input parameters for using latent Dirichlet allocation

When using topic modeling (Latent Dirichlet Allocation), the number of topics is an input parameter that the user need to specify. Looks to me that we should also provide a collection of candidate ...
5
votes
2answers
511 views

Algorithms for clustering documents by similar words and phrases

I'm working on a project where I'm trying to take a pair of documents and find and group (cluster) similar words and phrases between them. Which algorithm would solve this kind of a problem? I know ...
4
votes
2answers
627 views

How to plot results from text mining (e.g. classification or clustering)?

In text classification and clustering, the number of features are normally big, e.g. I currently get are around 5,000 features which is already really small compared to many other text mining tasks. ...
3
votes
0answers
113 views

Category selection for text classification

It is said that to achieve a good result (many different metrics) for text classification, it is not always a business of selecting the algorithm/classifier. Sometimes, it is even more important to ...
2
votes
0answers
59 views

Statistical analysis on categories before text classification

I want to classify text by different topics. However, one of the current problems is that there are several topics/categories that are quite intuitively independent and statistically standalone, but ...
3
votes
2answers
292 views

Clustering of 10's of millions of high dimensional data

I have a set of 50 million text snippets and I would like to create some clusters out of them. The dimensionality might be somewhere between 60k-100k. The average text snippet length would be 16 ...
0
votes
0answers
107 views

Understanding text Clustering

I have not had much luck explaining my question so please do make a comment if you don't understand what I am asking. Core Question: How can I approach the problem of clustering text[not text ...
5
votes
3answers
270 views

Semantic distance between excerpts of text

I'm wondering how far along the natural language processing is in determining the semantic distance between two excerpts of text. For instance, consider the following phrases Early today, I got up ...