84 votes

Choosing the right linkage method for hierarchical clustering

Methods overview Short reference about some linkage methods of hierarchical agglomerative cluster analysis (HAC). Basic version of HAC algorithm is one generic; it amounts to updating, at each step, ...
user avatar
  • 52.9k
74 votes
Accepted

How can an artificial neural network ANN, be used for unsupervised clustering?

Neural networks are widely used in unsupervised learning in order to learn better representations of the input data. For example, given a set of text documents, NN can learn a mapping from document to ...
user avatar
36 votes
Accepted

What are the differences between sparse coding and autoencoder?

Finding the differences can be done by looking at the models. Let's look at sparse coding first. Sparse coding Sparse coding minimizes the objective $$ \mathcal{L}_{\text{sc}} = \underbrace{||WH - ...
user avatar
  • 13k
24 votes

Evaluation measures of goodness or validity of clustering (without having truth labels)

An outline of internal clustering criteria (internal cluster validation indices) This is the excerpt from my documentation of a number of popular internal clustering criteria I've programmed, as a ...
user avatar
  • 52.9k
24 votes
Accepted

What's the intuition behind contrastive learning or approach?

Contrastive learning is very intuitive. If I ask you to find the matching animal in the photo below, you can do so quite easily. You understand the animal on left is a "cat" and you want to find ...
user avatar
22 votes

How can an artificial neural network ANN, be used for unsupervised clustering?

You want to look into self-organizing maps. Kohonen (who invented them) wrote a book about them. There are packages for this in R (som, kohonen), and there are implementations in other languages ...
user avatar
20 votes
Accepted

How to predict outcome with only positive cases as training?

This is called learning from positive and unlabeled data, or PU learning for short, and is an active niche of semi-supervised learning. Briefly, it is important to use the unlabeled data in the ...
user avatar
  • 17.6k
20 votes
Accepted

Choosing the hyperparameters using T-SNE for classification

I routinely use $t$-SNE (alongside clustering techniques - more on this in the end) to recognise/assess the presence of clusters in my data. Unfortunately to my knowledge there is no standard way to ...
user avatar
  • 36.2k
19 votes
Accepted

How can t-SNE or UMAP embed new (test) data, given that they are nonparametric?

Great question. I will answer it using t-SNE because I assume it is familiar to more people. I think UMAP is very promising and is a great contribution but to be honest I am getting a little bit ...
user avatar
  • 95.6k
19 votes
Accepted

Finding category with maximum likelihood method

This is a classic unsupervised learning problem that has a simple maximum likelihood solution. The solution is a motivating example for the expectation maximization algorithm. The process is: ...
user avatar
  • 53.6k
18 votes

Feature selection for clustering problems

I have a few thoughts to share about dimension reduction in unsupervised learning problems. In answering, I've assumed that your interest is in "high-touch," human involvement wrt cluster ...
user avatar
  • 9,842
17 votes

How to understand the drawbacks of Hierarchical Clustering?

Whereas $k$-means tries to optimize a global goal (variance of the clusters) and achieves a local optimum, agglomerative hierarchical clustering aims at finding the best step at each cluster fusion (...
user avatar
  • 3,958
16 votes

How do you learn labels with unsupervised learning?

Normally, you don't (and you don't believe everything someone writes somewhere on the internet). What the writer probably meant (at least that's my interpretation) is that you can use clustering to ...
user avatar
  • 6,433
15 votes
Accepted

How to choose an optimal number of latent factors in non-negative matrix factorization?

To choose an optimal number of latent factors in non-negative matrix factorization, use cross-validation. As you wrote, the aim of NMF is to find low-dimensional $\mathbf W$ and $\mathbf H$ with all ...
user avatar
  • 95.6k
15 votes
Accepted

Distinguishing between two groups in statistics and machine learning: hypothesis test vs. classification vs. clustering

Great question. Anything can be good or bad, useful or not, based on what your goals are (and perhaps on the nature of your situation). For the most part, these methods are designed to satisfy ...
user avatar
15 votes

Finding category with maximum likelihood method

What you are describing is a mixture of two Gaussians. $$ f(x) = \pi \, \mathcal{N}(\mu_1, \sigma_1^2) + (1 - \pi) \, \mathcal{N}(\mu_2, \sigma_2^2) $$ where $\pi \in (0, 1)$ is a mixing proportion. ...
user avatar
  • 117k
14 votes

How to understand the drawbacks of Hierarchical Clustering?

Scalability $k$ means is the clear winner here. $O(n\cdot k\cdot d\cdot i)$ is much better than the $O(n^3 d)$ (in a few cases $O(n^2 d)$) scalability of hierarchical clustering because usually both $...
user avatar
14 votes
Accepted

Are all Machine Learning algorithms divided into Classification and Regression, not just supervised learning?

All unsupervised algorithms, e.g. clustering, dimension reduction (PCA, t-sne, autoencoder,...), missing value imputation, outlier detection, ... Some of them might internally use regression or ...
user avatar
  • 10.7k
13 votes

Is overfitting a problem in unsupervised learning?

We talk about overfitting when the model performs better on training sample, then on validation sample. First of all, how would you define overfitting for unsupervised learning? If you conduct, say, ...
user avatar
  • 117k
12 votes

What are the differences between sparse coding and autoencoder?

In neuroscience the term Neural Coding is used to refer to the patterns of electrical activity of neurons induced by a stimulus. Sparse Coding by its turn is one kind of pattern. A code is said to be ...
user avatar
12 votes

Which unsupervised classification method can be used for categorical data?

I'm going to answer this as an approach to clustering categorical data. The standard k-means performs poorly in case of categorical data since in the sample space is discrete. The cost function ...
user avatar
12 votes
Accepted

Deriving Multiplicative Update Rules for NMF

$$\min_{W \in \mathbb{R}^{n \times k},H \in \mathbb{R}^{k \times m}} \left \| V- WH \right \|^{2}_F \text{ s.t. }W,H \geq 0 $$ $$\;\;\;\;\;\;Tr((V-WH)^T(V-WH)) \;\;\;\;\;\; \scriptsize \left [ \...
user avatar
12 votes

Are all Machine Learning algorithms divided into Classification and Regression, not just supervised learning?

No, it's much broader than that. You should at least read about the following: Clustering Dimensionality Reduction Reinforcement Learning
user avatar
  • 52.4k
11 votes

Distinguishing between two groups in statistics and machine learning: hypothesis test vs. classification vs. clustering

Not going to address clustering because it's been addressed in other answers, but: In general, the problem of testing whether two samples are meaningfully different is known as two-sample testing. ...
user avatar
  • 22.4k
11 votes
Accepted

Why use a Gaussian mixture model?

I'll borrow the notation from (1), which describes GMMs quite nicely in my opinon. Suppose we have a feature $X \in \mathbb{R}^d$. To model the distribution of $X$ we can fit a GMM of the form $$f(...
user avatar
  • 2,494
10 votes
Accepted

Is there any difference between distant supervision, self-training, self-supervised learning, and weak supervision?

There are two aspects to all the different terms you have given: 1] Process of obtaining training data 2] Algorithm that trains $f$ or the classifier The algorithm that trains $f$, regardless of how ...
user avatar
  • 3,592
10 votes
Accepted

t-SNE with mixed continuous and binary variables

Disclaimer: I only have tangential knowledge on the topic, but since no one else answered, I will give it a try Distance is important Any dimensionality reduction technique based on distances (tSNE, ...
user avatar
9 votes
Accepted

What is the minimum number of data points required for kernel density estimation?

In the book "Density Estimation for Statistics and Data Analysis, Bernard. W. Silverman, CRC ,1986" there is a chapter "Required sample size for given accuracy" where a sample size required is given ...
user avatar
  • 326
9 votes
Accepted

In convolutional neural network, what does fully-connected layer mean?

Every neuron from the previous layer is connected to every neuron on the next layer1.
user avatar
  • 3,105
9 votes

How to understand the drawbacks of Hierarchical Clustering?

I just wanted to add to the other answers a bit about how, in some sense, there is a strong theoretical reason to prefer certain hierarchical clustering methods. A common assumption in cluster ...
user avatar
  • 191

Only top scored, non community-wiki answers of a minimum length are eligible