0
votes
0answers
9 views

How to Hybrid the clustering and classification model [on hold]

Hi i am working on classification . By reading some papers on classification i found that the result of hybrid of clustering and classification provides better result. But, i donot know how to hybrid ...
1
vote
1answer
36 views

How does the Bayes' theorem equation generalize all sorts of regression/classification models?

I have been reading “Pattern Recognition & Machine Learning” written by Christopher M. Bishop for some time, but I am still a beginner. I wish to get a bigger view that summarizes regression and ...
1
vote
1answer
26 views

Algorithm for scoring co-varying traits

I am sure this has been done, but I can't find quite the right approach. EDIT: Trying to explain better. The rows of colored boxes below are columns of molecular sequence data -- positions in a ...
4
votes
2answers
91 views

What's the easiest way to separate two populations in a scatterplot?

I have to separate two populations by a line in a scatterplot: I would like find a threshold that separates the two populations. In @Waynes words, I would like to cluster the points into two ...
1
vote
1answer
49 views

Categorical variable with a very large number of categories as a predictor

I am trying to use a categorical variable as a predictor in a supervised learning setting, but there are too many categories for the classification algorithm to handle, something like over a 1000 ...
5
votes
1answer
107 views

Selecting an appropriate machine learning algorithm?

I do not think that this is a difficult question, but I guess someone needs experience to answer it. It is a question that is asked a lot here, but I did not found any answer that explains the reasons ...
2
votes
1answer
111 views

Which papers discuss classification or clustering of source code according to programming language?

My specific problem is to separate a huge archive of files containing source code and sometimes including embedded languages (apart from the main language).
0
votes
0answers
48 views

Clustering University Courses using Machine Learning

I have a database with 32344 Courses from Swedish universities. A course have the following attributes: ...
1
vote
1answer
46 views

Finding the best dataset for classification

I have 100 datasets. All of them have varying number of features. There are around 20,000 samples in each of them. Every $i$-th sample in the 100 datasets has the same label ($0/1$). The data is ...
1
vote
1answer
85 views

How to interpret these indices/metrics for comparing partitions intuitively out of these images?

Two sets of comparisons were performed between original clustering and the new clustering using several indices and metrics of performance. Below are the two initial clusterings or partitions (these ...
0
votes
0answers
19 views

Choose canonical values from clusters of erroneous ones

I suspect this is a statistical problem, but it may be just an algorithmic one. So, more formally: Given: a set C of unknown ‘canonical' values an error window e a set V of known values, s.t. ...
0
votes
0answers
36 views

Analyze which items contributed the most to a significant anova simple effect

I have a generic question that I am not even sure how to formulate, but: Imagine that two categorical factors, A and B, that are well-known to interact. A is manipulated as a between-subject factor ...
0
votes
1answer
107 views

Neighborhood analysis in Matlab using a dot plot

I have points in a 2D graph (coordinates: X,Y property: Z). I would like to find for every point the closest, for example, 5 points and save their properties. What would be the easiest approach? ...
1
vote
1answer
91 views

Sweeping across multiple classifiers and choosing the best?

I'm using Weka to perform classification, clustering, and some regression on a few large data sets. I'm currently trying out all the classifiers (decision tree, SVM, naive bayes, etc.). Is there an ...
2
votes
1answer
101 views

Cluster many thousands observations (mixed variable types). Cluster subsample and then classify the rest observations?

I'm trying to run a cluster analysis on a large dataset (70k+ observations to cluster) with mixed variables (numeric, ordinal, binary and nominal). I don't think I can create the distance matrix using ...
1
vote
1answer
117 views

Classification techniques on histograms

I have sets of different histograms that I wish to apply on them some classification techniques (e.g., PCA, LDA (linear discriminant analysis)) in order to cluster them. Therefore, what are the common ...
0
votes
2answers
162 views

Is it possible to use Hellinger distance for environmental variables?

Here is the problem, euclidean distance is not recommended for datasets with many zeroes (like matrices of species/site), as there is the risk of the abundance paradox (Orloci 1978). Whereas to ...
0
votes
0answers
103 views

Time series clustering or categorization

I am trying to figure out how to solve cluster or tag events. My data consists of many points in time. Each point has a time-stamp but only some (very few) of the events also have a label - let's say ...
0
votes
0answers
43 views

Feature extraction for processing

I have a parts of speech tagged corpus. I want to utilize this corpus as feature in clustering algorithm. Currently, scikit-learn supports(as far as I know), bag of words and tf-idf. Is it possible ...
1
vote
2answers
125 views

Similarity between objects based on tags (binary features)

I have five millions of objects each of them having one or more tags. How do I compute statistically sound similarity score between each pair of the objects taking into account that: There are 100 ...
0
votes
2answers
40 views

clustering gene expression data

I have a question about clustering. I' m managing gene expression microarray data and I would like to cluster them in classes. I searched around to find the best clustering algorithm for my data, ...
0
votes
1answer
203 views

Fit of a normal distribution to a one-dimensional dataset in R

I've got a set of (continuous) values from a measurement, where each object should be either positive or negative, and I know that the values of the "negative" objects should be approximately normally ...
1
vote
0answers
41 views

Using PCA to merge and grade correlated items

I have a real estates' condos sold dataset with the following fields DOM: Date on the market sellPct: Percentage difference between the original and final price. other fields such as Exposure( ...
0
votes
1answer
73 views

Using F1_score to measure cluster validity

I have clustered over 4000 textual files, and now I want to check and evaluate clusters. I want to use F-measure (a mix of recall and precision). The formal definition of F1_score is: $$ ...
8
votes
3answers
242 views

Detecting clusters in a binary sequence

I have a binary sequence such as 11111011011110101100000000000100101011011111101111100000000000011010100000010000000011101111 Where clusters of mostly 1's are ...
1
vote
1answer
132 views

Mahout Scability

Do you know any real world examples of how much Mahout can scale? I wonder how much it can scale in collaborative filtering, clustering, and classification ?
4
votes
2answers
295 views

How to define a posterior probability of y given x when the model is not probabilistic?

Suppose we have a very simple online k-means where each new data-point is assigned to its nearest center (the mean is updated incrementally). Each center (cluster) is labelled with the most common ...
4
votes
2answers
352 views

Classification on principal components

For my research I am doing classification on the dataset of three variables. I run unsupervised clustering (based on a histogram peak technique of cluster analysis)and the result I evaluated visually ...
2
votes
2answers
134 views

Choosing which data-point to label (active learning)

For an online unsupervised learning algorithm, data-points are learned sequentially. The performance may improve if in addition to the unlabelled data we have some labelled data-points (i.e. ...
0
votes
1answer
143 views

bag of words in an online configuration, for classification / clustering

I have a set of image documents. I extract text keywords from this images using OCR to represent each image as a bag of words (a vector where each value is the number of occurrence of a word in the ...
1
vote
1answer
84 views

Distance independent approximation of Nearest Neighbor/k-NN.

Nearest neighbor/k-NN for use with Normalized Compression Distance. I wonder if there exist any approximation of NN/k-NN algorithm which work for all distance measures ? I would like to test ...
1
vote
1answer
447 views

Clustering of time series

I have a set of almost 1600 time series on 2 years which I want to group into clusters. Do you think this is possible using k-means? Which method do you advice me to use? Is this possible at all using ...
-1
votes
1answer
132 views

Are Bayesian approches used for classification (supervised) or for clustering (unsupervised)?

Are Bayesian approaches (static and dynamic) used for classification (which is supervised) or for clustering (which is unsupervised)? or can they be used for both ? I even see that for instance to ...
6
votes
3answers
3k views

Supervised clustering or classification?

The second question is that I found in a discussion somewhere on the web talking about "supervised clustering", as far as I know, clustering is unsupervised, so what is exactly the meaning behind ...
2
votes
0answers
103 views

Strategies for Recovering Missing Data

I'm working on the following missing data problem to learn more about stats, probability, and machine learning, but I'm not really making progress solving it: I have a group of unordered, non-unique ...
6
votes
1answer
289 views

On cophenetic correlation for dendrogram clustering

Consider the context of a dendrogram clustering. Let us call original dissimilarities the distances between the individuals. After constructing the dendrogram we define the cophenetic dissimilarity ...
3
votes
2answers
124 views

How do I cluster data where some observation can not be in the same cluster?

Say I have 12 (x,y) positions: ...
1
vote
0answers
58 views

What is an appropriate classification method for analysis of down-hole geophysical data?

I am looking for a method that may be used to classify and 'cluster' some multi-dimensional data gathered from drill holes, to impute some (only partially measured) physical properties of the rocks ...
4
votes
4answers
1k views

Initializing K-means clustering

If I have a certain dataset, how smart would it be to initialize cluster centers using means of random samples of that dataset. For example, suppose I want 5 clusters. I take 5 random samples of say, ...
0
votes
1answer
176 views

Hierarchical clustering: is it possible to combine single-linkage clustering and average linkage clustering?

A "seismic section" shows amplitude for m discrete x values along its horizontal axis times n discrete time values along its vertical axis: Peaks in amplitude (black) are centered on horizons; ...
2
votes
1answer
136 views

Market segmentation based on a time of consumption

I'm an almost graduated applied math student. I do some sporadic work in marketing. I have done a few market segmentation projects. I am soon going to do one which is important to me. I usually ...
2
votes
2answers
197 views

Finding communities in online social networks by removing nodes

I want to carry out Graph Clustering in a huge undirected graph with millions of edges and nodes. Graph is almost clustered with different clusters joined together only by some nodes (kind of ...
4
votes
5answers
4k views

Is cosine similarity a classification or a clustering technique?

In document classification, is cosine similarity considered a classification or a clustering technique? But you need training data with the cosine similarity for creation of the centroid right?
1
vote
1answer
698 views

What are the differences between document classification and clustering when working with a single topic?

I am doing some web page clustering work and I'm going to use cosine similarity as my distance measure. Even though cosine similarity is a clustering technique, I have to give training data in order ...
-1
votes
1answer
211 views

Comparing two sets of pixels to determine whether they belong to the same object

I have two sets of data, and I want to know if the second set is sufficiently different from the first to be considered different. More specifically, I have a sample set A from a number of pixels in ...
2
votes
0answers
60 views

Regarding the size of training data for building classifier

When we build a classifier, like SVM or Naive Bayesian, are there any generic rules or theoretical derivations on the size of training data set? For example, to train a SVM-based classifier, what ...
5
votes
2answers
770 views

Algorithms for clustering documents by similar words and phrases

I'm working on a project where I'm trying to take a pair of documents and find and group (cluster) similar words and phrases between them. Which algorithm would solve this kind of a problem? I know ...
1
vote
1answer
361 views

Clustering short time series

I would like to classify a relatively large set (over 9000) of short times series. The length of each sequence varies, but I would say about 80 % has between 2 and 9 observations. While I could use a ...
0
votes
1answer
73 views

Choosing attributes for clustering/classification

The situation is as follows. There are 400 examples in the training set and 200 discrete classes (each class has two examples). There are a few thousand attributes. When I run dimensionality ...
4
votes
2answers
811 views

How to plot results from text mining (e.g. classification or clustering)?

In text classification and clustering, the number of features are normally big, e.g. I currently get are around 5,000 features which is already really small compared to many other text mining tasks. ...