Tagged Questions

Finding hidden (statistical) structure in unlabelled data, including clustering and feature extraction for dimensionality reduction.

learn more… | top users | synonyms

0
votes
0answers
15 views

EM soft clustering in lingpipe

In Lingpipe's EM tutorial they said that it is possible to run the algorithm with no supervised data: It is possible to train a classifier in a completely unsupervised fashion by having the ...
2
votes
0answers
28 views

Integrating Prior estimates in Simrank Model

I am reading SimRank paper by Jeh and Widom which tries to find the similarity between objects based on the relationships between them. Effectively, SimRank is a measure that says "two objects are ...
0
votes
0answers
20 views

Getting a posterior belief from noisy observations

Say I have a dataset $X = \{x_i\} \supset \mathbb{R}^n$ for which I assume that $x_i = y_i + \sigma(y_i)$ for some unobserved variables $\{ y_i \}$. That is, I believe that my data is subject to ...
1
vote
1answer
65 views

SVM confidence according to distance from hyperline

For a probabilistic multi-class classifier we can get probabilities of membership of a new point $x$ to each class $y_i$; in case of 3 classes suppose that we get $P(y_a|x) > P(y_b|x) > ...
1
vote
0answers
19 views

Grouping observations based on variables that sum to one

I have a problem where I am trying to group observations (most likely using k-means or a similar unsupervised learning tool) where each observation includes n-variables, with the total sum of these ...
2
votes
1answer
53 views

How can we get the confidence (or probability) that a data-point belongs to an unknown class?

Using any supervised classifier, we can usually get the probability that a data point $x$ belongs to each class $y_i$, i.e. $P(y_i|x)$. However, in the case where the data-point x may belong to none ...
1
vote
0answers
42 views

The expressive power of PoE over MoE

I am reading Hinton's paper "Training Products of Experts by Minimizing Contrastive Divergence" about training PoE. There is an explanation of PoE's expressiveness, which stats "because the posterior ...
0
votes
1answer
167 views

Distant supervision: supervised, semi-supervised, or both?

"Distant supervision" is a learning scheme in which a classifier is learned given a weakly labeled training set (training data is labeled automatically based on heuristics / rules). I think that both ...
2
votes
1answer
81 views

Is it necessary for a distance measure used in clustering to correspond to some valid vector space?

I have defined an distance measure based on some properties of points. But I'm not even sure that it corresponds to a valid distance in some vector space. Is this a necessary condition for clustering ...
2
votes
2answers
150 views

Comparing 2 classifiers with unlimited training data

I would like to compare 2 text classifiers C1 and C2, which can be trained with "unlimited" noisy training datasets, meaning that you can use as much data as you want for training, such data being ...
1
vote
0answers
132 views

Unsupervised anomaly detection with factor analysis (in R)

The basic idea i'm trying is to model the data with factor analysis, assuming a latent variable structure that underlies the observations. Labels for "real" anomalies are available and used for ...
2
votes
0answers
109 views

Computation of Maximization probabilities of the EM algorithm

I have implemented a semi-supervised Naive Bayes that makes use of the EM algorithm to iteratively learn from unlabeled data in a text classification problem, but I am not sure of the processing done ...
2
votes
2answers
114 views

Choosing which data-point to label (active learning)

For an online unsupervised learning algorithm, data-points are learned sequentially. The performance may improve if in addition to the unlabelled data we have some labelled data-points (i.e. ...
4
votes
0answers
122 views

Computation of log-likelihood in semi-supervised naive bayes

I have the following 2 questions about log-likelihood computation in semi-supervised Naive Bayes. I have read on several documents online that, in every EM iteration of the semi-supervised Naive ...
0
votes
2answers
39 views

Compute probability of a grouping being correct

I have an exemplar grouping of objects (each with their own feature vector) into categories. I am then given a new grouping of compeltely different objects, and Iw would like to compute the ...
1
vote
2answers
283 views

Image Clustering with K-means - Postprocessing

I did some clustering on an image (each pixel is an observation that has 5 variables associated with it), I get pretty detailed results but they are a little bit noisey... I think. I used K-means. ...
5
votes
2answers
112 views

How to measure shape of cluster?

I know that this question is not well defined, but some clusters tend to be elliptical or lie in lower dimensional space whilst the other have nonlinear shapes (in 2D or 3D examples). Is there any ...
2
votes
1answer
120 views

Which are the most effective clustering ensembles?

In supervised learning, there are some ensemble methods that overcome others significantly (adaboost or random forests to mention some). Few years later, also ensembles in unsupervised learning were ...
4
votes
2answers
104 views

What are features that distinguish clustering, blind signal separation and dimensionality reduction?

In terms of input -> [process] -> output what are features that distinguish clustering, blind signal separation and dimensionality reduction? From this ...
2
votes
1answer
613 views

K-means Mahalanobis vs Euclidean distance

I currently am trying to cluster "types" of changes on bitemporal multispectral satellite images. I applied a thing called a mad transform to both images, 5000 x 5000 pixels x 5 bands. Each band is a ...
4
votes
3answers
865 views

Supervised clustering or classification?

The second question is that I found in a discussion somewhere on the web talking about "supervised clustering", as far as I know, clustering is unsupervised, so what is exactly the meaning behind ...
1
vote
3answers
99 views

Can unsupervised evaluation measures for clustering replace a supervised evaluation measure?

Is it possible to have the same evaluation performances when comparing some clustering algorithms using many unsupervised evaluation measures instead of a supervised one ?
5
votes
1answer
246 views

Getting probabilities over 1 in positive and unlabeled learning

I have a question regarding PU-Learning, which deals with learning from positive-labeled (no labeled negative examples) and positive/negative-unlabeled data. Particularly, my question is about the ...
1
vote
1answer
169 views

Market / Customer Segmentation - Merging two different segmentations

I have a database where each observation is a person. They were questioned on their attitude towards the consumption of X category of product. I have being using K-means to segment this data. I have ...
4
votes
2answers
344 views

What is data blending?

This term appears frequently in the method-related threads. Is blending a specific method in data-mining and statistical learning? I cannot get a relevant result from google. It seems blending is ...
2
votes
2answers
259 views

Baum-Welch training example

I'm the author of a new Baum Welch trainer using MapReduce for the Apache Mahout project (https://issues.apache.org/jira/browse/MAHOUT-627) I'm looking for an example with a reasonably small data set ...
4
votes
2answers
390 views

Supervised approaches vs. topic models in sentiment analysis

I am researching Sentiment Analysis over social media, particularly classifying online texts such as blog posts as positive, negative or neutral. Most of the approaches I have found for sentiment ...
1
vote
0answers
101 views

Gaussian Mixture - Optimal number of components

So, getting an "idea" of the optimal number of clusters in k-means is well documented. I found an article on doing this in gaussian mixtures, but not sure I am convinced by it, don't understand it ...
4
votes
1answer
149 views

Cluster clickstream data

I've recently entered the realm of machine learning and a project I am working on requires me to cluster users based on the order they visited webpages on a website. I have data in the form of: ...
2
votes
1answer
135 views

K - means cluster always landing right on top of whole dataset mean

I have a so so sized data set - 30 000 observations. I would like to run K-means on them but to restrict the center(mean) of the data. This is, I would like to push the clusters away from this mean. ...
2
votes
2answers
191 views

Can you use discriminant analysis to classify new observations into categories generated by a previous $k$-means clustering?

After doing k-means clustering on a set of observations, I would like to construct a discriminant function so as to classify new observations into the categories I found after k-means. Is this at all ...
9
votes
2answers
3k views

How to produce a pretty plot of the results of k-means cluster analysis?

I'm using R to do K-means clustering. I'm using 14 variables to run K-means What is a pretty way to plot the results of K-means? Are there any existing implementations? Does having 14 variables ...
4
votes
4answers
597 views

Initializing K-means clustering

If I have a certain dataset, how smart would it be to initialize cluster centers using means of random samples of that dataset. For example, suppose I want 5 clusters. I take 5 random samples of say, ...
2
votes
1answer
113 views

Market segmentation based on a time of consumption

I'm an almost graduated applied math student. I do some sporadic work in marketing. I have done a few market segmentation projects. I am soon going to do one which is important to me. I usually ...
4
votes
2answers
163 views

List of likelihood-based classification techniques

This is a basic statistical pattern recognition question. I'm aware of LDA classification, Naive Bayes Classification techniques which give output as a likelihood (of data belonging to a certain ...
1
vote
2answers
394 views

Logging similarities between vectors with R

I'm trying to write a program that automatically groups similarities between vectors. The vectors are comprised of point coordinates. For example (assuming X, Y, and Z are numbers): Data Set 1: [1, ...
8
votes
2answers
359 views

Applying machine learning for DDoS filtering

In Stanford's Machine Learning course Andrew Ng mentioned applying ML in IT. Some time later when I got moderate size(about 20k bots) DDoS on our site I decided to fight against it using simple Neural ...
2
votes
2answers
329 views

Methods for comparing clustering results

I am doing an unsupervised clustering analysis for a genomics project. This means that I do not know when a particular clustering analysis is good or not. I am running different clustering algorithms ...
4
votes
1answer
329 views

Evaluation measure of clustering (without having truth labels)

I'm clustering a set of data but I don't have truth document that allow me to evaluate the result of clustering (I have unlabelled data), so I can not use an external evaluation measure. In this case, ...
1
vote
1answer
67 views

Convergence requirement of Parzen window estimates

In our Machine Learning class we recently came across Parzen Window Estimates. The following statement was made: Let $\hat p_n$ be the estimator $\hat p$ using $n$ data points and let $p (x)$ be the ...
3
votes
2answers
97 views

How to apply unsupervised classification to spatial data

I am trying to learn how to apply unsupervised classification to spatial data. In near infra red satelite pictures; the ocean is dark and the forest is white. Each pixel in such an image is an ...
1
vote
0answers
53 views

SOM automated/objective clustering

So as I understand it SOM is primarily a visualization tool and clustering is a logical next step after you construct a SOM from data. Typically, the clustering is subjective in that after looking at ...
8
votes
1answer
144 views

SOM clustering for nominal/circular variables

Just wondering if anyone is familiar with clustering nominal inputs. I've been looking at SOM as a solution but apparently it only works with numerical features. Are there any extensions for ...
6
votes
2answers
434 views

How to differentiate two subgroups from a histogram?

I have a set of samples in which I assume there are 2 definite subsets in it. I plotted their values in a histogram and found that there are two distinct modes as shown in the figure below. My ...
3
votes
1answer
285 views

How can I assess how descriptive feature vectors are?

I am assessing how good different features are for unsupervised classification of a set of objects. For each different feature I test, I have computed a feature vector that describes the object. I ...
13
votes
1answer
3k views

How to define number of clusters in K-means clustering?

Is there any way to determine the optimal cluster number or should I just try different values and check the error rates to decide on the best value?
2
votes
0answers
153 views

References on semi-supervised LDA

I'd like to perform semi-supervised LDA (Latent Dirichlet Allocation) in the following sense: I have several topics that I'd like to use, and have seed documents that relate to these topics. I'd like ...
3
votes
0answers
214 views

Recommended method for finding archetypes or clusters

I wish to cluster users together in a database, with each user represented by a number of features that are both discrete and continuous in nature. The aim is to define a small number of archetypal ...
4
votes
1answer
948 views

Generative vs discriminant models

What are the differences between generative and discriminant models (in the context of Bayesian learning and inference)? and what it is concerned with prediction, decision theory or unsupervised ...
1
vote
1answer
687 views

Soft and Hard EM (Expectation Maximization)

What is the difference between soft and hard expectation maximization? EDIT: ok, i've found out this paper: http://ttic.uchicago.edu/~dmcallester/ttic101-07/lectures/em/em.pdf that explain quite well ...

1 2