Partitioning data into subsets of objects according to their mutual "similarity," without using preexisting knowledge such as class labels. Clustered-standard-errors and/or cluster-samples should be tagged as such; do not use the "clustering" tag for them.
2
votes
1answer
134 views
Multiplicity in sample size calculation for stratified estimation problem
We want to determine the public opinion about a recently administered intervention in a health care network. There's a brief questionnaire of 10 Y/N questions in which we'd like to estimate ...
1
vote
0answers
51 views
What is an appropriate classification method for analysis of down-hole geophysical data?
I am looking for a method that may be used to classify and 'cluster' some multi-dimensional data gathered from drill holes, to impute some (only partially measured) physical properties of the rocks ...
0
votes
2answers
176 views
Distance function for categories in K-means
How to define a distance function when euclidean distance doesn't apply? For instance, say I have some data involves nationality. I'll probably assign a number to each nation, but for nations that ...
2
votes
0answers
294 views
Distance threshold for clustering
Usually online clustering methods (based on kmeans or not) define a distance threshold value. If a new data-point $x$ is far enough from the nearest center $c$ (i.e. the distance from $x$ to $c$ is ...
1
vote
1answer
82 views
What should I do while designing stratified cluster randomized sampling survey if cluster size is large/smaller than I expect?
I am design a sampling strategy for a health survey in long-term care facilities (LTCFs). I refer to the article here
and want to ask question about the cluster size.
I am thinking of doing a ...
3
votes
1answer
359 views
Ecological mixed data cluster analysis: Transformations required? Use K-means or hierarchical methods?
I am trying to identify habitat types from 85 plots. I intend to do a cluster analysis to identify habitat types, and hope to fit additional plots into the identified clusters.
(For context, I took ...
4
votes
1answer
76 views
How to do clustering on monthly collected data?
I want to cluster 10 different sites based upon different parameters like temperature, humidity, rain, etc. for monthly data collected over a period of 2 years.
Should I use means of all 24 data ...
4
votes
4answers
596 views
Initializing K-means clustering
If I have a certain dataset, how smart would it be to initialize cluster centers using means of random samples of that dataset. For example, suppose I want 5 clusters. I take 5 random samples of say, ...
1
vote
1answer
214 views
Quantitative evaluation metric of kmeans clustering results
I'm using k-means to cluster sentences according to the part-of-speech tags of the words in a sentence, and I have a nice, easy to understand visualization of the result, but I'm struggling to find a ...
7
votes
3answers
476 views
Mixture Models and Dirichlet Process Mixtures (beginner lectures or papers)
In the context of online clustering, I often find many papers talking about: "dirichlet process" and "finite/infinite mixture models".
Given that I've never used or read about dirichlet process or ...
0
votes
1answer
131 views
Hierarchical clustering: is it possible to combine single-linkage clustering and average linkage clustering?
A "seismic section" shows amplitude for m discrete x values along its horizontal axis times n discrete time values along its vertical axis:
Peaks in amplitude (black) are centered on horizons; ...
1
vote
3answers
312 views
Univariate clustering of time series
I just want to know if its possible to cluster an univariate time series, in order , say, to detect anomalies?
and do you have any online version for denstream code, in Matlab?
here is the time ...
8
votes
3answers
1k views
Reason to normalize in euclidean distance measures in hierarchical clustering
Apparently, in hierarchical clustering in which the distance measure is Euclidean distance, the data must be first normalized or standardized to prevent the covariate with the highest variance from ...
4
votes
2answers
283 views
Appropriateness of PCA to visualize clusters in genetic data
I've seen PCA improperly applied in genetic research quite often. I wanted to clarify : when is it appropriate to use PCA as a visualization tool in your analysis?
Some examples:
1) Rarely is the % ...
0
votes
3answers
710 views
Using a cosine similarity does not work for any dataset
I have a clustering algorithm, where if I use an euclidian distance as similarity, it works well on any dataset. If I replace it by a cosine similarity (see my code bellow), it will give a degenerate ...
2
votes
1answer
113 views
Market segmentation based on a time of consumption
I'm an almost graduated applied math student. I do some sporadic work in marketing. I have done a few market segmentation projects. I am soon going to do one which is important to me. I usually ...
0
votes
1answer
100 views
How can I replace this condition by a probability?
I want to see if a datapoint x should (or not) be assigned to a nearest component y using the following condition:
if ($d > T$) then {do not assign x to y}. With $d = distance(x,y)$ and $T = ...
2
votes
1answer
119 views
Nonnegative matrix factorization on a matrix with negative values
I have a gene expression profile with negative values.
Is there any way to do an Nonnegative matrix factorization (NMF) on a matrix which has a few negative values?
(I couldn't approximate the ...
1
vote
2answers
99 views
Clustering 5 dimensions into 4 based on mean?
In my project, one variable named Organizational Culture has 5 dimensions namely Employee Development, Harmony, Customer Orientation, Social Responsibility and Innovation. In SPSS, I need to configure ...
14
votes
5answers
1k views
Euclidean distance is usually not good for sparse data?
I have seen somewhere that classical distances (like Euclidean distance) become weakly discriminant when we have multidimensional and sparse data. Why? Do you have an example of two sparse data ...
2
votes
2answers
326 views
What if k-means starts in a local minimum?
I have to find 10 clusters of 100 samples with dimension 100. I have access to two k-means implementations. Both of them initialize the means with 10 randomly picked samples. When I run these ...
5
votes
3answers
295 views
Weighted clustering algorithm
I am looking to divide the 50 US states into n regions. The requirements in dividing are:
Each state will be assigned a value
The state values in each region ...
2
votes
2answers
152 views
Finding communities in online social networks by removing nodes
I want to carry out Graph Clustering in a huge undirected graph with millions of edges and nodes. Graph is almost clustered with different clusters joined together only by some nodes (kind of ...
4
votes
2answers
160 views
Problem of “clustering” into most similar groups
I do not think that the following problem can be solved with k-means clustering. I am not sure though. Okay, let me describe the problem. I need to find a way or an algorithm that groups members of a ...
1
vote
0answers
288 views
Cubic clustering criterion in R
Does anybody know if any package calculates the cubic clustering criterion (CCC) index in R to aid the selection of optimal number of clusters?
3
votes
1answer
147 views
How to decide if to do dimensionality reduction before clustering?
Is there any agreement on when to reduce data dimension before clustering in order to avoid curse of dimensionality?
My intuition is that if I have say 1000 points and data dimension is 10 then it is ...
0
votes
2answers
263 views
Clustering based on distance measure between data [duplicate]
Possible Duplicate:
Clustering with a distance matrix
I have a set of data which I wish to cluster.
I have computed a distance measure between each pair of data, but I am limited in that I ...
0
votes
1answer
119 views
Can clustering be used for log-odds scores?
I have data which looks like this. As you can see the data is symmetric and its not exactly a distance matrix. They are log odds ratios. And the diagonal values are higher than non-diagonal elements.
...
1
vote
0answers
111 views
Correcting standard errors when the independent variables are autocorrelated
I have a question about how to correct standard errors when the independent variable has correlation. In a simple time series setting we can use Newey-West covariance matrix with a bunch of lags and ...
1
vote
1answer
171 views
clustering with particle filters
Suppose we want to cluster a data stream of unknown number of clusters, and estimate them using particle filters. With particle filters, we need to know $P(x_t | x_{t-1})$ and $P(z_t | x_t)$ (where z ...
1
vote
1answer
154 views
Maximum entropy inference for k-means clustering
I am taking a course on Maximum Entropy Inference (MEI), where its application to k-Means was discussed. I am confused about the problem setting.
As far as I understand, our goal is to find ...
2
votes
2answers
214 views
Clustering using the BIRCH algorithm
I am trying to solve this question:
Build a CF-tree for the subset of points, (3,3) (4,3) (6,3) (7,4) (7,5)
assuming that the branching factor, B, is set to 2, the maximum number of sub-clusters at ...
2
votes
1answer
818 views
Dirichlet process/Chinese restaurant process for clustering in R
I recently read a fascinating article describing methods for clustering data without assuming a fixed number of clusters.
The article even includes some sample code, in a mix of Ruby, Python, and R. ...
1
vote
2answers
198 views
Clustering time series based on correlation
I want to cluster a set of time series regarding their pairwise correlation. If I normalize the series by subtracting their average value and then scaling to a standard deviation of one, the ...
4
votes
1answer
144 views
Struggling to understand the relationship between two definitions of entropy
I am familiar with entropy as defined from information theory, which is simply
$ H(X) = \sum_{i=1}^{n} p(x_{i}) \log p(x_{i}) $,
for a discrete set of probabilities. In term of clustering, the ...
0
votes
0answers
63 views
How do I do group wise clustering in R?
How do I do group wise clustering in R?
Hi all,
I have N x K data matrix, where N is the number of observations, K is the number of variables.
The N observations fall into M categories or groups.
...
4
votes
5answers
3k views
Is cosine similarity a classification or a clustering technique?
In document classification, is cosine similarity considered a classification or a clustering technique? But you need training data with the cosine similarity for creation of the centroid right?
1
vote
1answer
386 views
What are the differences between document classification and clustering when working with a single topic?
I am doing some web page clustering work and I'm going to use cosine similarity as my distance measure. Even though cosine similarity is a clustering technique, I have to give training data in order ...
1
vote
1answer
117 views
Characterizing the inter-arrival time of software threads
For a multi-threaded application, I want to identify the nature of the application based on the arrival times of each thread. Example(Are thread launch spaced regularly, are they bursty in nature or ...
4
votes
3answers
774 views
Clustering a dataset with both discrete and continuous variables
I have a dataset X which has 10 dimensions, 4 of which are discrete values.
In fact, those 4 discrete variables are ordinal, i.e. a higher value implies a higher/better semantic.
2 of these discrete ...
2
votes
2answers
81 views
How to determine variables that make certain cluster different from others?
Say clustering was performed. My concern is to understand what characterizes certain cluster. Which variables differ most for a certain cluster? Are there any methods for performing such kind of ...
-1
votes
1answer
134 views
Comparing two sets of pixels to determine whether they belong to the same object
I have two sets of data, and I want to know if the second set is sufficiently different from the first to be considered different.
More specifically, I have a sample set A from a number of pixels in ...
0
votes
1answer
94 views
Cluster analysis with ties issue
When I perform cluster analysis in SAS, the SAS log sometimes return a warning something like this:
...
1
vote
2answers
394 views
Logging similarities between vectors with R
I'm trying to write a program that automatically groups similarities between vectors. The vectors are comprised of point coordinates.
For example (assuming X, Y, and Z are numbers):
Data Set 1: [1, ...
2
votes
2answers
200 views
Predicting SSE in k-means clustering
Given any number of clusters, is it possible to estimate the Sum of Squares Error (SSE) for the Clusters after adding noise to the clustering?
The type of noise generated will be supplied as a ...
0
votes
0answers
31 views
Labeled multidimensional sequences
I'm trying to find similarities in 3-dimensional sequential data.
The sequences are 3-uples $(t,r,d)$ each sequence is generated by one subject during a 3-6 months period:
$t$ is a task ...
1
vote
1answer
136 views
Cluster analysis of boolean vectors in R
I have 114 vectors with 6 boolean attributes. I saw that might be several distinct clusters in a simple visualization. K-means clustering on the transformed vectors (true = 1, false = 0) results in ...
1
vote
3answers
364 views
Cluster quality measures
Does Matlab provide any facility for evaluating clustering methods? (cluster compactness and cluster separation. ....)
Or is there any toolbox for it?
3
votes
3answers
260 views
Mixture model fixed effects
One standard model used with panel data is fixed effects: $y_{it} = \mu_i + \theta_t + \epsilon_{it}$, where $i$ is the individual and $t$ is time subscripts. This can be estimated easily with OLS and ...
1
vote
1answer
109 views
Clustering similar values in a matrix
I posted this on stackoverflow.com yesterday and its had very few views. I came across this stackexchange site and thought its got to be worth an ask:
I have an interesting problem and I'm sure ...
