Tagged Questions

0
votes
2answers
57 views

Can I replenish my dataset with unlabeled data?

I have a database of time series data generated by giroscope sensor. A small part of the data is already labeled and used to fit the model. My strong assumption is that most of the samples left in ...
3
votes
0answers
26 views

How to combine multiple similarity measures?

I have a hyperspectral image where the pixels are 21 channels. So each pixel $\in \mathbb{R}^{21}$. I want to perform clustering on the pixels with similarity defined by two different measures, one ...
0
votes
0answers
45 views

Clustering-based Identification

I am trying to find a way to merge data that correspond to users and have the form $user_i, property_1, property_2, .\dots, property_n$, where $n\geq50$, $i\geq 10^6$ We might have missing data in ...
0
votes
2answers
31 views

clustering gene expression data

I have a question about clustering. I' m managing gene expression microarray data and I would like to cluster them in classes. I searched around to find the best clustering algorithm for my data, ...
0
votes
2answers
60 views

Time Series Similarity : Differing Lengths with R

I am experimenting with creating a distance matrix between time series for clustering and similarity searching. The main reference I am using is for the Similarity procedure in SAS (Paper). I would ...
0
votes
1answer
48 views

Clustering a dataset to get the most abnormal data [duplicate]

I have several datasets in R+, each containing two training and test sets. For example the following dataset. I want to train a classifier by using training data such that by applying the test data, I ...
0
votes
0answers
59 views

How to identify a new pattern in a URL with a machine learning algorithm (Text mining)

I am trying to identify new patterns after analyzing a number of URLs. So let's say, I am investigating the hypothetical website Yoohle.com and their URLs have the following structure. domain = ...
1
vote
1answer
111 views

How can one show a Kmeans solution is unique?

Suppose we are given a distribution P and a constant K. We wish to minimize the kmeans objective w.r.t centers ${C1,..Ck}$: What constraints on $P$ are known to imply that the optimal solution is ...
1
vote
1answer
37 views

Generating rules to obtain a given categorical distribution… is it possible?

I'm working on a problem, I was wondering if there are any methods available to do the following. I have a data set with information on people (continuous and categorical data). I have 3 categories ...
2
votes
1answer
216 views

Kmeans on “symmetric” data

A set is said to be fully-symmetric if for every x in it, negating one of its components results in y such that y is in the set as well. A set is said to be semi-symmetric if for every x in it, ...
1
vote
1answer
45 views

Non-exclusive categorical model

I have a project to model users with characteristic tags (e.g. runner, cyclist, swimmer, vegan, pianist) in order to correlate user behaviour to these labels. Obviously a user can have multiple ...
1
vote
2answers
67 views

Clustering and variable selection

Let's say I'm trying to cluster $n$ points in $\mathbb{R}^p$, and I know in advance that only $s$ many of these $p$ dimensions determine the differences between the clusters. Of course, I don't know ...
2
votes
0answers
57 views

Appropriate threshold to map a similarity value to an edge in a graph

In order to cluster users given a user-item binary matrix data, I am planning to first find user's similarity (Jaccard) and then use graph theory to isolate clusters (communities). I need to map the ...
2
votes
1answer
83 views

Is it necessary for a distance measure used in clustering to correspond to some valid vector space?

I have defined an distance measure based on some properties of points. But I'm not even sure that it corresponds to a valid distance in some vector space. Is this a necessary condition for clustering ...
1
vote
1answer
55 views

Shape analysis of an object to create features for pattern recognition

I am currently with a medical imaging project. Just wondering how to measure the shape of a sphere. For example, how to give a measurement that an object is more like a sphere than the other? I know ...
0
votes
0answers
24 views

Analysis of Systematic differences in two multifactor pricing models

I am trying to compare two different pricing models for a product. The two models take the same inputs ( 10-12 different factors)and i know the definitions of both functions which calculate the price. ...
2
votes
2answers
116 views

Choosing which data-point to label (active learning)

For an online unsupervised learning algorithm, data-points are learned sequentially. The performance may improve if in addition to the unlabelled data we have some labelled data-points (i.e. ...
3
votes
3answers
230 views

How to decide whether to reuse old code or reinvent the wheel?

For a long time now, I have been thinking about working with neural networks and genetic algorithms. I have never been able to decide whether it makes sense to start writing my own code, or to reuse ...
0
votes
2answers
41 views

Compute probability of a grouping being correct

I have an exemplar grouping of objects (each with their own feature vector) into categories. I am then given a new grouping of compeltely different objects, and Iw would like to compute the ...
7
votes
5answers
247 views

Does preclustering help to build a better predictive model?

For the task of churn modelling I was considering: Compute k clusters for the data Build k models for each cluster individually. The rationale for that is,that there is nothing to prove, that the ...
0
votes
0answers
312 views

Mahout k-means generates too many clusters [closed]

I am a beginner in Mahout, I use Mahout 0.8 and followed the tutorial in https://cwiki.apache.org/MAHOUT/clustering-of-synthetic-control-data.html When I use : ...
2
votes
2answers
163 views

Finding most informative feature subsets given dataset, clustering algorithm and gold standard partition

I have an $n \times m$ matrix of data $\mathbf{D}$ as well as a $k$-partition $P$ of $n$ indices each representing a row in $\mathbf{D}$. Assuming an arbitrary clustering algorithm $A$, I would like ...
9
votes
5answers
310 views

Appropriate clustering techniques for temporal data?

I have temporal data of activity frequencies. I want to identify clusters in the data that indicate distinct periods of time with similar activity levels. Ideally I want to identify the clusters ...
2
votes
1answer
148 views

Alternative to Otsu for dividing data into two groups

I need to be able to automatically divide a dataset into two clusters. There are heuristic reasons to expect the data to have two clusters which would be visually clear if one were to plot the data ...
2
votes
1answer
149 views

Fuzzy C-means and its stages of clustering

when dealing with soft and hard clustering techniques such as K-means and fuzzy C-means I run into abit of difficulty on the steps that FCM takes to calculate the clusters. For instance in K-means ...
1
vote
2answers
275 views

Hybrid (K-means + Hierarchical ) clustering

I have a huge dataset (50,000 2000-dimensional sparse feature vectors). I want to cluster them in to k (unknown)clusters. As hierarchical clustering is very expensive in terms of time complexity ...
0
votes
1answer
123 views

Can clustering be used for log-odds scores?

I have data which looks like this. As you can see the data is symmetric and its not exactly a distance matrix. They are log odds ratios. And the diagonal values are higher than non-diagonal elements. ...
4
votes
5answers
3k views

Is cosine similarity a classification or a clustering technique?

In document classification, is cosine similarity considered a classification or a clustering technique? But you need training data with the cosine similarity for creation of the centroid right?
0
votes
1answer
98 views

Cluster analysis with ties issue

When I perform cluster analysis in SAS, the SAS log sometimes return a warning something like this: ...
3
votes
1answer
530 views

The input parameters for using latent Dirichlet allocation

When using topic modeling (Latent Dirichlet Allocation), the number of topics is an input parameter that the user need to specify. Looks to me that we should also provide a collection of candidate ...
6
votes
3answers
941 views

Comparing clusterings: Rand index vs variation of information

I was wondering if anybody had any insight or intuition behind the difference between the Variation of Information and the Rand Index for comparing clusterings. I have read the paper "Comparing ...
1
vote
2answers
102 views

Clustering with probabilities

I will ask my question beginning by an example (I am novice in stats) I have a set of probabilities from a given observation A={0.3,0.2,0.001, 0.02, ...} I want to partition A on subsets or clusters ...
1
vote
0answers
260 views

Performance of the fuzzy c-means clustering algorithm

I have implemented a genetic algorithm for a fuzzy c-means clustering in Matlab. Its performance should be apriori better than that of the classic fuzzy c-means (fcm function in matlab). However, on ...
5
votes
3answers
249 views

Books about incremental data clustering

Does anyone have a suggestion of any relatively recent and good book about data clustering? More specifically, I'm looking for incremental clustering.
3
votes
1answer
94 views

Growing hierarchical self-organizing maps

For learning GHSOM I figured out I should study SOM as a first step. Now, I know the basics of SOM, about weighted vectors and euclidean distances: when the human brain cannot easily process more ...
1
vote
1answer
124 views

Identifying culs-de-sac (circular housing arrangements) with GIS data

I'm thinking about a new project, so I don't have data yet, but I plan on collecting GIS information for houses within a state. Usually in the U.S., these dead-end streets will have a large circle ...
8
votes
7answers
1k views

What are the “hot algorithms” for machine learning?

This is a naive question from someone starting to learn machine learning. I'm reading these days the book "Machine Learning: An algorithmic perspective" from Marsland. I find it useful as an ...
3
votes
0answers
115 views

Category selection for text classification

It is said that to achieve a good result (many different metrics) for text classification, it is not always a business of selecting the algorithm/classifier. Sometimes, it is even more important to ...
2
votes
0answers
61 views

Statistical analysis on categories before text classification

I want to classify text by different topics. However, one of the current problems is that there are several topics/categories that are quite intuitively independent and statistically standalone, but ...
3
votes
2answers
401 views

Validation of clustering results

I have a data which contains several columns which I later reduced using a PCA algorithms to two different components. I then applied the k-means algorithms to the data. Now, how can I verify that my ...
7
votes
3answers
363 views

Are there cases where there is no optimal k in k-means?

This has been inside my mind for at least a few hours. I was trying to find an optimal k for the output from the k-means algorithm (with a cosine similarity metric) so I ended up plotting the ...
2
votes
3answers
306 views

Usage of LDA with more than two classes

I'm reading about the Linear Discriminant Analysis by Fisher and I have a couple of questions about its usage. If you have k>2 classes in a two-dimensional space you find k−1 vectors that you need ...
4
votes
1answer
282 views

Interpretation of a one cluster solution using the EM cluster algorithm

I'm trying to use the EM cluster algorithm, provided by the software Weka, to classify my data and it only finds one cluster. Could I interpret this as there are no ways to distinguish the ...
7
votes
1answer
2k views

Clustering: Should I use the Jensen-Shannon Divergence or its square?

I am clustering probability distributions using the Affinity Propagation algorithm, and I plan to use Jensen-Shannon Divergence as my distance metric. Is it correct to use JSD itself as the distance, ...
85
votes
8answers
13k views

Detecting a given face in a database of facial images

I'm working on a little project involving the faces of twitter users via their profile pictures. A problem I've encountered is that after I filter out all but the images that are clear portrait ...
8
votes
5answers
2k views

Clustering genes in a time course experiment

I have seen a few queries on clustering in time series and specifically on clustering, but I don't think they answer my question. Background: I want to cluster genes in a time course experiment in ...
6
votes
4answers
907 views

Recommended books or articles as introduction to Cluster Analysis?

I'm working on a small (200M) corpus of text, which I want to explore with some cluster analysis. What books or articles on that subject would you recommend?
1
vote
1answer
115 views

For data similar to audio, how to determine if there are 1 or 2 categories? [closed]

I have to compare pairs of audio strems as 1d time series. Looking at the aligned trajectories I need to either cluster them together or assume they arise from independent generators. I remember ...