Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

This may seem a trivial question but I have a fundamental problem in understanding learning.

  1. In supervised learning, given and input-output pair, what are we learning? Are we learning the inputs (say features) or the output? For example, In terms of linear system, let a system be y=Ax ; y is the output and x is the input. What is learning in this case?

  2. In unsupervised learning, what are we learning since no outputs are known.

All information that is available says that learning means finding weights. But what are we actually is not clear to me. In general what are we learning?

Shall appreciate if the doubts are cleared.

share|improve this question
Neither, we are learning A. Weight can be thought of as the entries of x being weighted by the columns of A. In unsupervised learning we are in some sense learning x. Think of clustering. We are saying what can we say about the structure of x so that some x values are closer than others –  Sid Jul 30 at 5:49
Can you please elaborate as an asnwer?I thought A is the weight matrix. In the example, where inputs are say situations and for each input we have the outcome as actions like pick, drop, move; what is being learnt so that the desired actions can be performed? –  Ria George Jul 30 at 5:56

1 Answer 1

up vote 7 down vote accepted

We are learning a model to describe the relationship between inputs ($\mathbf{X}$) and output(s) ($\mathbf{y}$). What this means exactly depends on the nature of the model: it can be coefficients/weights, rules, ...

In supervised learning, we try to learn the relationship between inputs $\mathbf{X}$ and outputs $\mathbf{y}$. In general: $$\mathbf{y} = f(\mathbf{X}),$$ where $f(\cdot)$ is some unknown function that we want to learn (e.g. approximate its behaviour).

Consider a standard linear model, for which: $$\mathbf{y} \approx \mathbf{\hat{y}} = \mathbf{\beta}\mathbf{X}$$

The outputs $\mathbf{y}$ and inputs $\mathbf{X}$ are known. This is what we call training data. There is nothing to learn there. We try to learn how they are related to eachother as well as possible, which is the model (in this case $\mathbf{\beta}$).

Typically as well as possible means we want to optimize some kind of measure, for example maximizing the classification accuracy or minimizing the mean squared error of predictions $\mathbf{\hat{y}}$ compared to the truth $\mathbf{y}$.

In unsupervised learning we only try to describe the data at our disposal (e.g. $\mathbf{X}$). Clustering, for instance, attempts to retrieve sets of patterns that are similar to each other in some way.

share|improve this answer
Thank you for your reply. Can you please address these doubts?(A)When inputs are features and outputs are classes, I thought we are learning the class for each feature. Since in classification we want to classifiy which pattern it belongs to(B) consider example in robot:Inputs are different scenarios like situations(S) and the desired output(O) is the response to each situation in the form of actions like pick,drop.Is robot learning when to perform the action(Output) so that when a situation occurs it can know what it should be doing?So,is it not learning O so as to respond to an unknown (S)? –  Ria George Jul 30 at 7:18
(A) for classification the output of the function we want to learn $f(\cdot)$ are class labels, (B) you try to build a model to learn the mapping $S\mapsto O$ and then use it to predict an optimal output for a new S. –  Marc Claesen Jul 30 at 8:05

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.