What are supervised learning and unsupervised learning from a connectionist point of view

Question

The general concept of supervised learning and unsupervised learning is very clear.

In supervised learning, the decision on the unlabeled data is done after learning a classifier using available training samples, as examples of supervised classifiers we have decision tree, neural network, support vector machine(SVM).

Whereas, in an unsupervised system, the classifier does not have any labeled sample. In this later case, the classification is done by exploiting some criteria like Euclidean distance, a common example of the unsupervised classification method is the k-means cluster classifier.

But what are these learning algorithms in connectionist?

Answer 1

For clarity, I make a distinction between classification models and regression models.

• A classification problem is when a statistical model (the classifier) is being trained to predict the category of a feature vector (can also be a feature matrix, like an image or sub-image). Take for example recognition of hand-written letters of postal codes, on all letters in the daily mail. As not every person has a clear hand writing, a few mistakes are being made so now and then. The output of the classifier is the probability of each numeric character (0,1,2,...).
• A regression problem is when a statistical model (the regresssion model) is being trained to predict a real-numbered variable, based on a feature vector. Imagine trying to predict tomorrows opening figure of the Down Jones stock index, based on variables known today. The predicted number is the conditional mean, which is also known as the expected value (of the future Dow Jones index height).

Supervized and unsupervized learning algorithms have been applied to classification problems over the last several decades. A supervized, connectionist approach is the feed-forward neural network. The target outcomes are known at forehand, and the neural network is trained to predict the target outcomes.

An unsupervized connectionist approach is Hebbian learning. After training, the Hebbian learned neural network associates a new (noisy) input with the most similar category. Many more unsupervized algorithms have been developed over the years. Take for example the self-organizing map and independent component analysis. They are both unsupervized algorithms, which can be trained to perform a kind of classification task, based on closest resemblense of the input pattern.

Regression problems are in most cases approached by supervized algorithms.