0
votes
0answers
14 views

Feature Selection for look alike modeling using k-NN

I have a list of items(around 7000) and various parameters for each items. For each item on my list i need to identify 10 items which are similar to the item from my whole population (17 million). I ...
2
votes
1answer
31 views

How can I evaluate the performance of a system that generates word clusters?

The word2vec tool uses deep learning to compute vector representations of words. They've mentioned that - "The word vectors can be also used for deriving word classes from huge data sets. This is ...
0
votes
1answer
32 views

Performing hierarchical clustering on a large data set

I have been applying complete linkage on about 5,000 points using matlab with no problem. I want to extend this method to much more elements. It would take me a long time to process my data to test ...
0
votes
1answer
21 views

Evaluation indexes hypothesis for clustering

I read on the cluster analysis page of wikipedia: For example, k-means clustering can only find convex clusters, and many evaluation indexes assume convex clusters. On a data set with non-convex ...
0
votes
0answers
10 views

k-way MinMax spectral clustering

Is there a k-way MinMax Cut Spectral Clustering which can be easily implemented? In the spectral clustering tutorial I found only 2-way MinMax cut.
0
votes
0answers
27 views

What is the best algorithm for finding attacks from log file [closed]

I m working on forensic analysis of web logs. I have generated the DoS attack dataset and i m having the attack dataset of log files (unlabeled dataset) taken from Dr. Anton Chuvakin. I need to look ...
1
vote
0answers
19 views

Comparison of cophenetic correlation coefficients on different data sets

I applied the same hierarchical clustering (weighted) on two data sets: The first is a 'raw' data set, on which I didn't do anything, and the second on the same data set after I filtered it by ...
0
votes
0answers
18 views

processing a sound file for analysis of different spoken languages

So i have sound files for 5 languages by 2 person, thus my input data has 10 sound files. Now i want to cluster them based on the languages (thus 5 clusters) and not based on the speaker/voice ( ...
0
votes
0answers
23 views

Converting a spearman correlation to a euclidian dissimilarity

I am applying ward hierarchical clustering on a data set for which I have pairwise similarities. Since hierarchical clustering need a dissimilarity matrix, I am trying to convert my similarity matrix ...
1
vote
0answers
33 views

Understanding and Implementing a Dirichlet Process model

I am trying to implement and learn a Dirichlet Process to cluster my data (or as machine learning people speak, estimate the density). I read a lot of paper in the topic and sort of got the idea. ...
0
votes
0answers
17 views

What is data augmented by the additive inverse?

I am reading Biclustering of expression data (Cheng and Church, 2000) The paper is about the Cheng and Church biclustering algorithm and its main metric, the mean squared residue (MSR). It is said ...
0
votes
0answers
29 views

Converting a similarity to a dissimilarity [duplicate]

I am working on a clustering problem for which I have to manually choose the number of clusters. I have a visualization tool that helps me decide whether the clusters are good. In order to ...
1
vote
0answers
22 views

What is a shift bicluster?

I am reading A comparative analysis of biclustering algorithms for gene expression data (Eren, Kemal, et al. - 2013) When explaining the Cheng and Church method, it says that: MSR was shown to be ...
1
vote
0answers
28 views

Vector Quantization of heavy tailed distribution

I'm generating with Monte Carlo simulation some stock price $X$. Once I have the stock price sample, I want to cluster it with 100 points $\hat{X}$. My problem is that the error associate with my ...
0
votes
1answer
24 views

Data Conversion to Standard data format in hierarchical Dirichlet process

I'm trying to test the performance of posterior inference on a set of documents with hierarchical Dirichlet process for topic modeling. How can i convert my data (document) to standard data format ...
-1
votes
1answer
19 views

Converting FuzzykMeans to SphericalFuzzyKMeans?

I grabbed an implementation of FuzzyKMeans (FuzzyCMeans) from the nightly build of the Apache Commons Math library, but I now realize I need to use Cosine Similarity instead of the Euclidean Distance. ...
-1
votes
1answer
45 views

Clustering algorithms assigning probability values

I have a distance matrix for some data I want to cluster. However, I don't just want to assign elements to clusters, but I also want to assign a probability for each element to belong to each cluster. ...
2
votes
1answer
85 views

Run time analysis of the clustering algorithm (k-means)

I was reading some notes on ML and clustering and it claimed that the run time of clustering was O(kn) where k is the number of clusters and n is the number of points. I was wondering why this was ...
2
votes
1answer
60 views

Evaluating the clustering of a Kohonen UMatrix

Given a converged Kohonen feature map, how would one evaluate the clustering in terms of intra- and inter-cluster distances? Assuming that both the trained codebook vectors and Unified Distance ...
0
votes
0answers
23 views

Tuning the inflation parameter in mcl

I read on the mcl documentation that the inflation parameter can be used to tune the granularity of the clusters. I am not very familiar with graph theory. What is the granularity of the clusters? Can ...
0
votes
3answers
60 views

Converting a distance to a similarity

I am working on a graph clustering algorithm (mcl). It gives the opportunity to give weights to the edges. The weights must be similarities, but I have a distance. The values of this distance range ...
2
votes
0answers
27 views

Choice of an evaluation metric for a graph clustering algorithm

I have instances for which the only thing I know is 70% of the distance matrix. I know some of these points form groups of correlated points (each point of a group is "close" to every point of the ...
3
votes
1answer
33 views

gaussian mixture model - approximate a matrix

I have a similarity matrix M - the value M(i,j) indicates the similarity between two elements i and j. I want to approximate that matrix using a Gaussian Mixture model or I want to cluster that ...
2
votes
2answers
139 views

Using the gap statistic to compare algorithms

I want to compare the performances of two clustering algorithms that give me different numbers of clusters. I recently learned about the gap statistic. However, from what I have learned, this ...
0
votes
1answer
39 views

How to define silhouette for one cluster?

I want to compare two clustering algorithms. I took data that the first algorithm gathered in one cluster. The second algorithm gave 3 clusters for the same points. In order to compare the results, I ...
0
votes
3answers
91 views

Scalability of Markov Clustering

I want to do graph clustering on a large dataset (A graph with 600,000 Nodes and tens of millions of edges). I read about Markov clustering. I saw this algorithm involved the calculation of a ...
-2
votes
1answer
42 views

k-medoids algorithm with incomplete distance matrix

I want to apply k-medoids algorithm using an incomplete distance matrix as input. How can I handle the lack of information of this matrix? Just ignoring the missing distances? Or is there a better ...
1
vote
1answer
34 views

k-core clustering algorithm

I am trying to cluster data. Each point in this dataset is connected to some other points. I want to define clusters "depending on how much the points are connected to each other". After some ...
1
vote
0answers
33 views

Random initialization with k-means clustering

I read on my machine learning course (on coursera) that random initialization performed several times and then taking the cluster with the lowest cose could help when the number of clusters is ...
0
votes
1answer
112 views

Multidimensional scaling using Python

I have 6,000 points for which I have all pairwise distances in a distance matrix. I want to get an idea whether these data were generated by a mixture of Gaussian distributions so I'm trying to get a ...
5
votes
1answer
108 views

Selecting an appropriate machine learning algorithm?

I do not think that this is a difficult question, but I guess someone needs experience to answer it. It is a question that is asked a lot here, but I did not found any answer that explains the reasons ...
4
votes
3answers
152 views

Can you compare different clustering methods on a dataset with no ground truth by cross-validation?

Currently, I am trying to analyze a text document dataset that has no ground truth. I was told that you can use k-fold cross validation to compare different clustering methods. However, the examples I ...
0
votes
0answers
12 views

Increase Recall Rate for SIFT

Is there a better way to increase Recall Rate when using SIFT features? I am thinking a way to replace the NN1/NN2 ratio to account for slightly distorted objects. Moving towards clustering and using ...
2
votes
0answers
41 views

Expectation Maximization Clarification

I found very helpful tutorial regarding EM algorithm. The example and the picture from the tutorial is simply brilliant. Related question about calculating probabilities how does expectation ...
1
vote
1answer
79 views

Dirichlet process mixture model with Bayesian hierarchical clustering

I am doing Bayesian hierarchical clustering. From my understanding, there are three basic points for this algorithm. Use marginal likelihoods to decide which clusters to merge Asks what the ...
0
votes
0answers
48 views

Clustering University Courses using Machine Learning

I have a database with 32344 Courses from Swedish universities. A course have the following attributes: ...
1
vote
2answers
92 views

Why we build Laplacian graph in spectral clustering?

Dose anybody know's what creating Laplacian Graph from similarity matrix brings us in spectral clustering ? or why we create it ? Here it's the Algorithm: Laplacian graph is : L= D-W. ,D: degree ...
3
votes
1answer
96 views

Keyword clustering

I have one million of keywords (from search queries in google), and I need to group them semantically. I have already done some research and I have found information about how to extract keywords and ...
1
vote
2answers
82 views

Relationship between dimentionality reduction and clustering algorithms

I've got bit confused about dimensionality reduction and clustering . whether all clustering algorithms (k-means, affinity propagation, spectral clustering,...) do kind of dimensionality reduction ?
0
votes
0answers
21 views

Scaled graph laplacian in presence of loops

I am interested in spectral clustering so I was looking the scikit-learn code for computing the Lapacian of a graph given its weighted adjacency matrix. ...
2
votes
1answer
138 views

Performance metrics to evaluate unsupervised learning

With respect to the unsupervised learning (like clustering), are there any metrics to evaluate performance?
1
vote
0answers
35 views

What is exactly code vector and quantization vector of self organizing map?

I am trying to understand code vector in self organizing map. Could anybody explain me intuitively what it is exactly?
2
votes
1answer
109 views

Estimating most important features in a k-means cluster partition

Is there a way to determine which features/variables of the dataset are the most important/dominant within a kmeans cluster solution generated via R?
0
votes
0answers
36 views

self organizig map with spectral clustering

I'm looking for implementation of self organizing map which for the clustering part uses spectral clustering rather than k-means. Does anyone knows something about it ?
0
votes
0answers
79 views

Use matrix feature for machine learning or cluster analysis

I have a bunch of features that I would like to use for classification/machine learning and cluster analysis. Normally I use single point values or transformations of values for features and ...
0
votes
2answers
102 views

Machine learning algorithms/approaches for class recommendations?

I am asking a theoretical question about machine learning in terms of clustering. Is it possible, given a set of data of classes that students have taken in a semester to recommend additional classes ...
0
votes
0answers
15 views

cluster - class dependency bag of words [duplicate]

I have three clusters which is obtained by clustering features from images. Lets assume I have two classes. Consider the following table, where c denotes the class number and k denotes the cluster ...
1
vote
1answer
108 views

Computing mutual information

I have a problem when computing the mutual information between two variables. Let's consider the following table: ...
1
vote
1answer
91 views

Sweeping across multiple classifiers and choosing the best?

I'm using Weka to perform classification, clustering, and some regression on a few large data sets. I'm currently trying out all the classifiers (decision tree, SVM, naive bayes, etc.). Is there an ...
0
votes
1answer
87 views

Quantitative results of clustering analysis

Currently, I am doing a clustering analysis for two sets of data. One smaller dataset (about 100 data) got ground truth labels, and one larger dataset (about 2000 data) has no ground truth labels. ...