I got nice graphical representation of Machine learning for clustering / classification.

enter image description here

Source: Kernel Approaches to Unsupervised and Supervised Machine Learning by Sun-Yuan Kung

Here are my thoughts on difference:

Supervised vs Unsupervised:

Supervised learning is the machine learning task of inferring a function from labeled training data. The training data consist of a set of training examples. In supervised learning, each example is a pair consisting of an input object (typically a vector) and a desired output value (also called the supervisory signal). A supervised learning algorithm analyzes the training data and produces an inferred function, which can be used for mapping new examples.

In machine learning, the problem of unsupervised learning is that of trying to find hidden structure in unlabeled data. Since the examples given to the learner are unlabeled, there is no error or reward signal to evaluate a potential solution.

Both of these methods can be classified in linear and non-linear Kernels.

Approaches to unsupervised learning include: clustering (e.g., k-means, mixture models, hierarchical clustering), hidden Markov models,blind signal separation using feature extraction techniques for dimensionality reduction (e.g., principal component analysis, independent component analysis, non-negative matrix factorization, singular value decomposition).

Neutral network artificial neural networks (ANNs) are computational models inspired by an animal's central nervous systems (in particular the brain) which is capable of machine learning as well as pattern recognition. Artificial neural networks are generally presented as systems of interconnected "neurons" which can compute values from inputs.

Among neural network models, the self-organizing map (SOM) and adaptive resonance theory (ART) are commonly used unsupervised learning algorithms. The SOM is a topographic organization in which nearby locations in the map represent inputs with similar properties. The ART model allows the number of clusters to vary with problem size and lets the user control the degree of similarity between members of the same clusters by means of a user-defined constant called the vigilance parameter. ART networks are also used for many pattern recognition tasks, such as automatic target recognition and seismic signal processing.

My questions are as follows:

  1. What is difference between Euclidean-Tree and Kernel-Tree ? or Euclidean-K-Means SOM vs Kernel version?

  2. Difference between SVM vs Neural Network or SOM?

  3. Difference between Correlation and Kernel Matrices ?


Where did you get this tree from? It appears to be highly nonstandard, including terminology. No wonder you are confused...

I have never heard the terms "Euclidean Tree", "Kernel Tree". I can only guess this refers to hierarchical clustering using Euclidean distance and kernelized distance.

For clustering I strongly advise against using machine learning literature. To ML people, everything looks like classification; but this isn't a classification task.

SVM and SOM are something completely different. Similar, your other questions mostly indicate you need to read more literature: these questions cannot be answered as is. You are essentially asking "What is the difference between Apples and Oranges".

  • 1
    $\begingroup$ I don't understand your comment regarding the ML approach to clustering. It is not even remotely formulated as a classification problem. $\endgroup$ – Marc Claesen Aug 3 '14 at 12:38
  • $\begingroup$ @Anony-Mousse See the link in recent edits source of the chart. I do admit I need more reading but I need some directions, thanks $\endgroup$ – John Aug 3 '14 at 13:36
  • $\begingroup$ I highly recommend the FREE text book, The Elements of Statistical Learning $\endgroup$ – bdeonovic Aug 3 '14 at 13:57
  • $\begingroup$ @MarcClaesen its nevertheless usually focused around rediscovering what is already known in the data, instead of discovering some new structure. In the broadest sense, you cannot even "formulate" the clustering problem. If you do so (e.g. k-means) you put up some major constraints. K-means is a variance minimization problem, for example. It's not really "discovering structure in the data". It's all about minimizing L2 errors. $\endgroup$ – Anony-Mousse Aug 3 '14 at 21:48
  • 1
    $\begingroup$ @Anony-Mousse I don't follow your reasoning ... because there's some minimization involved it's not 'proper' clustering? How do you feel about (kernel) spectral clustering? What do you consider valid clustering approaches? $\endgroup$ – Marc Claesen Aug 3 '14 at 22:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.