0
votes
0answers
9 views

Mean of vectors minimizes L2-norm [duplicate]

Recently introduced to data mining and am trying to understand how the mean of vectors minimizes L2-Norm/Euclidean distance. I've tried googling it and can't seem to find a proof as to how something ...
0
votes
2answers
23 views

How to propositionalize a relational data set for clustering analysis?

I am working with a data set of students and their courses for a single semester, attempting to cluster based on the courses & various other attributes where "courses" are the "many" side of a ...
0
votes
0answers
8 views

automatic assign class name based on text

My question is , I have a set of plain text , i want to create category based on the text. Eg: i have written something about Soup recepie then the algorithm must create a category called Food. After ...
0
votes
2answers
29 views

Suggestion for discovering inherent patterns in data

I have a a big data set of clients with all sorts of variables that describe their background, payment history, and more... I also have a subset of those client who all have portrayed similar ...
1
vote
3answers
31 views

Computing Image Similarity based on Color Distribution

Image Similarity based on Color Palette Distribution I am trying to compute similarity between two images based on their color palette distribution, let's say I have two sets of key value pairs as ...
1
vote
1answer
55 views

Different hierarchical clustering results

I'm running a hierarchical clustering on a sample of data using the steps below: ...
1
vote
1answer
46 views

The representation of a high-dimensional data set by a low number of data points

I know that some of the questions I am asking here have been answered in a general case in the two questions I am referring to in the problem section. Nonetheless, I am asking for a very specific case ...
-1
votes
2answers
44 views

How can I get an overview of realtime dataset?

Can I use some part of the real time dataset for getting an overview about the dataset before applying an algorithm? Can I use ELKI or ...
1
vote
2answers
140 views

How to perform K-medoids when having the distance matrix

I've been trying for a long time to figure out how to perform (on paper)the K-medoids algorithm, however I'm not able to understand how to begin and iterate. for example: I have the distance matrix ...
0
votes
1answer
66 views

Performing hierarchical clustering on a large data set

I have been applying complete linkage on about 5,000 points using matlab with no problem. I want to extend this method to much more elements. It would take me a long time to process my data to test ...
0
votes
1answer
25 views

Evaluation indexes hypothesis for clustering

I read on the cluster analysis page of wikipedia: For example, k-means clustering can only find convex clusters, and many evaluation indexes assume convex clusters. On a data set with non-convex ...
1
vote
0answers
29 views

Comparison of cophenetic correlation coefficients on different data sets

I applied the same hierarchical clustering (weighted) on two data sets: The first is a 'raw' data set, on which I didn't do anything, and the second on the same data set after I filtered it by ...
0
votes
0answers
86 views

Converting a spearman correlation to a euclidian dissimilarity

I am applying ward hierarchical clustering on a data set for which I have pairwise similarities. Since hierarchical clustering need a dissimilarity matrix, I am trying to convert my similarity matrix ...
2
votes
0answers
32 views

Principles to create natural data for a clustering algorithm?

I have come up with an algorithm to cluster geospatial data points. I have quite a few volunteers (100) collecting data for me, using my app on their smart phones to check in at places. However, I'm ...
1
vote
0answers
26 views

Clustering Techniques

I'm a little new to data mining and would definitely appreciate some tips. I'm using clustering algorithms looking for possible grouping in some variables described below. I've been using the Excel ...
0
votes
0answers
29 views

Converting a similarity to a dissimilarity [duplicate]

I am working on a clustering problem for which I have to manually choose the number of clusters. I have a visualization tool that helps me decide whether the clusters are good. In order to ...
-1
votes
1answer
26 views

Converting FuzzykMeans to SphericalFuzzyKMeans?

I grabbed an implementation of FuzzyKMeans (FuzzyCMeans) from the nightly build of the Apache Commons Math library, but I now realize I need to use Cosine Similarity instead of the Euclidean Distance. ...
-2
votes
1answer
55 views

Clustering algorithms assigning probability values

I have a distance matrix for some data I want to cluster. However, I don't just want to assign elements to clusters, but I also want to assign a probability for each element to belong to each cluster. ...
1
vote
0answers
28 views

Regression clustering

I am looking for references about classical methods in regression clustering. My problem is the following: I have a cloud of points that are assumed to have been generated by inverse functions with ...
0
votes
0answers
84 views

Clustering email with mixed types of attribute

I am looking to cluster thousands of emails in one's mailbox. Different from traditional analysis with emphasis on email body, the attachments will play a big role in my work. The data set contains ...
0
votes
0answers
44 views

Tuning the inflation parameter in mcl

I read on the mcl documentation that the inflation parameter can be used to tune the granularity of the clusters. I am not very familiar with graph theory. What is the granularity of the clusters? Can ...
0
votes
3answers
71 views

Converting a distance to a similarity

I am working on a graph clustering algorithm (mcl). It gives the opportunity to give weights to the edges. The weights must be similarities, but I have a distance. The values of this distance range ...
2
votes
0answers
35 views

Choice of an evaluation metric for a graph clustering algorithm

I have instances for which the only thing I know is 70% of the distance matrix. I know some of these points form groups of correlated points (each point of a group is "close" to every point of the ...
3
votes
2answers
183 views

Using the gap statistic to compare algorithms

I want to compare the performances of two clustering algorithms that give me different numbers of clusters. I recently learned about the gap statistic. However, from what I have learned, this ...
0
votes
1answer
64 views

How to define silhouette for one cluster?

I want to compare two clustering algorithms. I took data that the first algorithm gathered in one cluster. The second algorithm gave 3 clusters for the same points. In order to compare the results, I ...
1
vote
3answers
260 views

Scalability of Markov Clustering

I want to do graph clustering on a large dataset (A graph with 600,000 Nodes and tens of millions of edges). I read about Markov clustering. I saw this algorithm involved the calculation of a ...
-2
votes
1answer
60 views

k-medoids algorithm with incomplete distance matrix

I want to apply k-medoids algorithm using an incomplete distance matrix as input. How can I handle the lack of information of this matrix? Just ignoring the missing distances? Or is there a better ...
1
vote
1answer
55 views

k-core clustering algorithm

I am trying to cluster data. Each point in this dataset is connected to some other points. I want to define clusters "depending on how much the points are connected to each other". After some ...
1
vote
0answers
38 views

Random initialization with k-means clustering

I read on my machine learning course (on coursera) that random initialization performed several times and then taking the cluster with the lowest cose could help when the number of clusters is ...
2
votes
2answers
348 views

How to summarize and understand the reults of DBSCAN clustering on big data?

Many clustering algorithms can be used with big data, eg. versions of KMeans, DBSCAN based on Hadoop, etc. But, with k means we will get k centroids for k clusters and we can map them to the space and ...
1
vote
1answer
126 views

Clarification needed about min/sim hashing + LSH

I have a reasonable understanding of the technique to detect similar documents consisting in first computing their minhash signatures (from their shingles, or n-grams), and then use an LSH-based ...
1
vote
0answers
68 views

Difference between BIRCH and Two-Step Clustering

I don't have SPSS. But judging from the documentation you can find on the internet, SPSS "Two-Step Clustering" seems to be closely related to BIRCH clustering: Zhang, T., Ramakrishnan, R., & ...
1
vote
3answers
1k views

Clustering with Weka

I have to analyse a data set with weka clustering, using 3 clustering algorithms and I need to provide a comparison between them about their performance and suitability. The comparison may include a ...
4
votes
1answer
535 views

Performance metrics to evaluate unsupervised learning

With respect to the unsupervised learning (like clustering), are there any metrics to evaluate performance?
0
votes
1answer
25 views

Could Robust Subspace Clustering be used to remove outliers?

I have two samples set, one is positive, and the other is negative. But, in each of both sets, there are some outliers that don't belong to it. Can the Robust subspace clustering be used to help me ...
1
vote
1answer
113 views

Sweeping across multiple classifiers and choosing the best?

I'm using Weka to perform classification, clustering, and some regression on a few large data sets. I'm currently trying out all the classifiers (decision tree, SVM, naive bayes, etc.). Is there an ...
1
vote
1answer
158 views

Clustering of documents that are very different in number of words

I have a corpus of 643 documents with different sizes and my goal is to cluster them according their topics and label each cluster with semantic name for its main topic. I have tired different ...
1
vote
1answer
461 views

Computing document similarity in latent semantic analysis

I have a question regarding Latent Semantic Analysis - after performing SVD decomposition of term-document matrix and choosing some number of dimensions, I get the set of new document vectors. Now, ...
0
votes
2answers
62 views

Clustering cloud fitting inverse functions

I have a cloud of points that can be clustered so that each set of points in the cluster can be fitted with an inverse function : $f(x) = cte / x$. What would be an approach for clustering my dataset ...
3
votes
2answers
2k views

LSA vs. PCA (document clustering)

I'm investigation various techniques used in document clustering and I would like to clear some doubts concerning PCA (principal component analysis) and LSA (latent semantic analysis). First thing - ...
-1
votes
1answer
466 views

Normalizing Term Frequency for document clustering

I have a problem understanding the normalization of Term Frequency weight in document Vector Space Model for clustering. Let's say that for document d I have counted occurences of all terms. I ...
5
votes
1answer
2k views

Difference between standard and spherical k-means algorithms

I would like to understand, what is the major implementation difference between standard and spherical k-means clustering algorithms. In each step, k-means computes distances between element vectors ...
1
vote
1answer
120 views

Distances vs. “distance like functions” in clustering

I am studying Kogan's "Introduction to Clustering Large and High Dimensional Data" because I would like to better understand clustering (I never worked with it). Until now "clustering" means to me to ...
0
votes
1answer
267 views

F-measure for document clustering evaluation - NaN

I'm developing the Java application for text document clustering, and I'm researching some evaluation methods. I implemented F-measure (http://en.wikipedia.org/wiki/F1_score), but I have a problem - ...
1
vote
1answer
72 views

Clustering spatio-temporal data?

I have data in the form of timestamp,lat,long which is gps data for users. I'm new to data mining and want to understand how can I start clustering these data to understand more about it. Should I ...
2
votes
2answers
1k views

Why do we use k-means instead of other algorithms?

I researched about k-means and these are what I got: k-means is one of the simplest algorithm which uses unsupervised learning method to solve known clustering issues. It works really well with large ...
1
vote
0answers
45 views

Using PCA to merge and grade correlated items

I have a real estates' condos sold dataset with the following fields DOM: Date on the market sellPct: Percentage difference between the original and final price. other fields such as Exposure( ...
1
vote
0answers
93 views

How to identify a new pattern in a URL with a machine learning algorithm (Text mining)

I am trying to identify new patterns after analyzing a number of URLs. So let's say, I am investigating the hypothetical website Yoohle.com and their URLs have the following structure. domain = ...
8
votes
3answers
307 views

Detecting clusters in a binary sequence

I have a binary sequence such as 11111011011110101100000000000100101011011111101111100000000000011010100000010000000011101111 Where clusters of mostly 1's are ...
5
votes
1answer
183 views

Binary classification of DNA motif sequences (bioinformatics)

I've been working on on a method for binary classification of DNA sequences. In more detail, here is what the method does. Given a family of DNA sequences, for example DNA sequence motifs, I try to ...