Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [unsupervised-learning]

Finding hidden (statistical) structure in unlabelled data, including clustering and feature extraction for dimensionality reduction.

76
votes
3answers
148k views

How to produce a pretty plot of the results of k-means cluster analysis?

I'm using R to do K-means clustering. I'm using 14 variables to run K-means What is a pretty way to plot the results of K-means? Are there any existing implementations? Does having 14 variables ...
48
votes
2answers
77k views

How can an artificial neural network ANN, be used for unsupervised clustering?

I understand how an artificial neural network (ANN), can be trained in a supervised manner using backpropogation to improve the fitting by decreasing the error in ...
37
votes
1answer
52k views

Performance metrics to evaluate unsupervised learning

With respect to the unsupervised learning (like clustering), are there any metrics to evaluate performance?
31
votes
4answers
28k views

What are the differences between sparse coding and autoencoder?

Sparse coding is defined as learning an over-complete set of basis vectors to represent input vectors (<-- why do we want this) . What are the differences between sparse coding and autoencoder? ...
29
votes
5answers
4k views

Distinguishing between two groups in statistics and machine learning: hypothesis test vs. classification vs. clustering

Assume I have two data groups, labeled A and B (each containing e.g. 200 samples and 1 feature), and I want to know if they are different. I could: a) perform a statistical test (e.g. t-test) to see ...
29
votes
2answers
32k views

Supervised learning, unsupervised learning and reinforcement learning: Workflow basics

Supervised learning 1) A human builds a classifier based on input and output data 2) That classifier is trained with a training set of data 3) That classifier is tested with a test set of data 4) ...
28
votes
2answers
38k views

Choosing the right linkage method for hierarchical clustering

I am performing hierarchical clustering on data I've gathered and processed from the reddit data dump on Google BigQuery. My process is the following: Get the latest 1000 posts in /r/politics ...
26
votes
4answers
26k views

Evaluation measures of goodness or validity of clustering (without having truth labels)

I'm clustering a set of data but I don't have truth document that allow me to evaluate the result of clustering (I have unlabelled data), so I can not use an external evaluation measure. In this case, ...
26
votes
3answers
24k views

Unsupervised, supervised and semi-supervised learning

In the context of machine learning, what is the difference between unsupervised learning supervised learning and semi-supervised learning? And what are some of the main algorithmic approaches to ...
22
votes
3answers
36k views

Supervised clustering or classification?

The second question is that I found in a discussion somewhere on the web talking about "supervised clustering", as far as I know, clustering is unsupervised, so what is exactly the meaning behind "...
21
votes
2answers
12k views

Generative vs discriminative models (in Bayesian context)

What are the differences between generative and discriminative (discriminant) models (in the context of Bayesian learning and inference)? and what it is concerned with prediction, decision theory or ...
21
votes
3answers
3k views

How to predict outcome with only positive cases as training?

For the sake of simplicity, let's say I'm working on the classic example of spam/not-spam emails. I have a set of 20000 emails. Of these, I know that 2000 are spam but I don't have any example of not-...
21
votes
5answers
11k views

Clustering procedure where each cluster has an equal number of points?

I have some points $X=\{x_1,...,x_n\}$ in $R^p$, and I want to cluster the points so that: Each cluster contains an equal number of elements of $X$. (Assume that the number of clusters divides $n$.) ...
19
votes
2answers
6k views

What is the manifold assumption in semi-supervised learning?

I am trying to figure out what the manifold assumption means in semi-supervised learning. Can anyone explain in a simple way? I cannot get the intuition behind it. It says that your data lie on a low-...
19
votes
1answer
17k views

How to define number of clusters in K-means clustering?

Is there any way to determine the optimal cluster number or should I just try different values and check the error rates to decide on the best value?
17
votes
1answer
12k views

Distant supervision: supervised, semi-supervised, or both?

"Distant supervision" is a learning scheme in which a classifier is learned given a weakly labeled training set (training data is labeled automatically based on heuristics / rules). I think that both ...
15
votes
4answers
18k views

How to understand the drawbacks of Hierarchical Clustering?

Can someone explain the pros and cons of Hierarchical Clustering? Does Hierarchical Clustering have the same drawbacks as K means? What are the advantages of Hierarchical Clustering over K means? ...
15
votes
3answers
8k views

What is data blending?

This term appears frequently in the method-related threads. Is blending a specific method in data-mining and statistical learning? I cannot get a relevant result from google. It seems blending is ...
14
votes
4answers
10k views

Is there “unsupervised regression”?

If I am correct, "unsupervised classification" is same as clustering. Then is there "unsupervised regression"? Thanks!
14
votes
3answers
482 views

What *is* an Artificial Neural Network?

As we delve into Neural Networks literature, we get to identify other methods with neuromorphic topologies ("Neural-Network"-like architectures). And I'm not talking about the Universal Approximation ...
14
votes
3answers
8k views

How to choose an optimal number of latent factors in non-negative matrix factorization?

Given a matrix $\mathbf V^{m \times n}$, Non-negative Matrix Factorization (NMF) finds two non-negative matrices $\mathbf W^{m \times k}$ and $\mathbf H^{k \times n}$ (i.e. with all elements $\ge 0$) ...
13
votes
3answers
3k views

Choosing the hyperparameters using T-SNE for classification

In as specific problem that I work with (a competition) I have the follwoing setting: 21 features (numerical on [0,1]) and a binary output. I have approx 100 K rows. The setting seems to be very noisy....
13
votes
4answers
20k views

Initializing K-means centers by the means of random subsamples of the dataset?

If I have a certain dataset, how smart would it be to initialize cluster centers using means of random samples of that dataset? For example, suppose I want ...
12
votes
4answers
5k views

Can you compare different clustering methods on a dataset with no ground truth by cross-validation?

Currently, I am trying to analyze a text document dataset that has no ground truth. I was told that you can use k-fold cross validation to compare different clustering methods. However, the examples I ...
12
votes
2answers
6k views

Applying machine learning for DDoS filtering

In Stanford's Machine Learning course Andrew Ng mentioned applying ML in IT. Some time later when I got moderate size(about 20k bots) DDoS on our site I decided to fight against it using simple Neural ...
12
votes
4answers
1k views

How to measure shape of cluster?

I know that this question is not well defined, but some clusters tend to be elliptical or lie in lower dimensional space whilst the other have nonlinear shapes (in 2D or 3D examples). Is there any ...
11
votes
5answers
2k views

SOM clustering for nominal/circular variables

Just wondering if anyone is familiar with clustering nominal inputs. I've been looking at SOM as a solution but apparently it only works with numerical features. Are there any extensions for ...
10
votes
2answers
2k views

How to understand a convolutional deep belief network for audio classification?

In "Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations" by Lee et. al.(PDF) Convolutional DBN's are proposed. Also the method is evaluated for image ...
9
votes
2answers
10k views

Feature selection for clustering problems

I am trying to make group together different datasets using unsupervised algorithms (clustering). The problem is that I have many features (~500) and a small amount of cases (200-300). So far I used ...
9
votes
1answer
2k views

Cannot make this autoencoder network function properly (with convolutional and maxpool layers)

Autoencoder networks seems to be way trickier than normal classifier MLP networks. After several attempts using Lasagne all what I get in the reconstructed output is something that resembles at its ...
8
votes
1answer
9k views

SVM confidence according to distance from hyperline

For a probabilistic multi-class classifier we can get probabilities of membership of a new point $x$ to each class $y_i$; in case of 3 classes suppose that we get $P(y_a|x) > P(y_b|x) > P(y_c|x)$...
8
votes
6answers
4k views

How to prepare/construct features for anomaly detection (network security data)

My goal is to analyse network logs (e.g., Apache, syslog, Active Directory security audit and so on) using clustering / anomaly detection for intrusion detection purposes. From the logs I have a lot ...
8
votes
2answers
2k views

Why only the mean value is used in (K-means) clustering method?

In clustering methods such as K-means, the euclidean distance is the metric to use. As a result, we only calculate the mean values within each cluster. And then adjustments are made on the elements ...
8
votes
2answers
2k views

Cluster clickstream data

I've recently entered the realm of machine learning and a project I am working on requires me to cluster users based on the order they visited webpages on a website. I have data in the form of: ...
8
votes
1answer
2k views

Self organizing maps vs. kernel k-means

For an application, I want to cluster data (potentially high dimensional) and extract probability of belonging to a cluster. I consider at the moment Self organizing maps or kernel k-means to do the ...
8
votes
1answer
3k views

Optimal number of components in a Gaussian mixture

So, getting an "idea" of the optimal number of clusters in k-means is well documented. I found an article on doing this in gaussian mixtures, but not sure I am convinced by it, don't understand it ...
7
votes
2answers
712 views

Why use a Gaussian mixture model?

I am learning about Gaussian mixture models (GMM) but I am confused as to why anyone should ever use this algorithm. How is this algorithm better than other standard clustering algorithm such as $K$-...
7
votes
2answers
1k views

Comparing 2 classifiers with unlimited training data

I would like to compare 2 text classifiers C1 and C2, which can be trained with "unlimited" noisy training datasets, meaning that you can use as much data as you want for training, such data being ...
7
votes
2answers
4k views

In convolutional neural network, what does fully-connected layer mean?

There are convolution layers, pooling layers, and possibly a classifier layer (e.g. softmax layer) in a convolutional neural network (CNN). I heard that there is also a fully-connected layer. What ...
7
votes
2answers
4k views

What is the fastest unsupervised feature learning algorithm?

I took a look at several unsupervised feature learning algorithms. Most of them (restricted Boltzmann machines and sparse auto-encoders) have very long training times even on small datasets like MNIST....
7
votes
3answers
2k views

Sequential pattern mining on single sequence

Can someone give me a hint about a good approach to find a frequent patterns in a single sequence. For example there is the single sequence ...
7
votes
2answers
6k views

Supervised approaches vs. topic models in sentiment analysis

I am researching Sentiment Analysis over social media, particularly classifying online texts such as blog posts as positive, negative or neutral. Most of the approaches I have found for sentiment ...
7
votes
1answer
3k views

When to use LDA over GMM for clustering?

I have a dataset containing user activity with 168 dimensions, where I want to extract clusters using unsupervised learning. It is not obvious to me whether to use a topic modelling approach in Latent ...
7
votes
1answer
629 views

Getting probabilities over 1 in positive and unlabeled learning

I have a question regarding PU-Learning, which deals with learning from positive-labeled (no labeled negative examples) and positive/negative-unlabeled data. Particularly, my question is about the ...
7
votes
1answer
4k views

How do I implement a deep autoencoder?

I'm trying to replicate results of this paper using Theano. The problem at the moment is, all Theano-related tutorials are only for MNIST classifiers, which isn't much use in unsupervised image ...
7
votes
3answers
993 views

Why is the restricted boltzmann machine both unsupervised and generative?

The restricted boltzmann machine is a generative learning model - but it is also unsupervised? A generative model learns the joint probability P(X,Y) then uses ...
6
votes
4answers
659 views

t-SNE dimensions as additional predictor variables

This question could also (maybe) relate to PCA. I built a supervised RandomForest on a dataset that I'm currently working on - the actual V Prediction $R^2$ was holding around 80% across many CV ...
6
votes
2answers
2k views

Constructing features from k-means

I would like to construct features from the result of apply k-means clustering to construct features of my data that I can later use for a classifier. Assume I have fixed the $k$ (e.g. 5) and ...
6
votes
2answers
1k views

Deriving Multiplicative Update Rules for NMF

How to derive the multiplicative update rules for the non-negative matrix factorization problem given by Lee and Seung. Minimize $\left \| V - WH \right \|^2$ with respect to $W$ and $H$, subject ...