# What is being learnt?

This may seem a trivial question but I have a fundamental problem in understanding learning.

1. In supervised learning, given and input-output pair, what are we learning? Are we learning the inputs (say features) or the output? For example, In terms of linear system, let a system be y=Ax ; y is the output and x is the input. What is learning in this case?

2. In unsupervised learning, what are we learning since no outputs are known.

All information that is available says that learning means finding weights. But what are we actually is not clear to me. In general what are we learning?

Shall appreciate if the doubts are cleared.

• Neither, we are learning A. Weight can be thought of as the entries of x being weighted by the columns of A. In unsupervised learning we are in some sense learning x. Think of clustering. We are saying what can we say about the structure of x so that some x values are closer than others
– Sid
Jul 30, 2014 at 5:49
• Can you please elaborate as an asnwer?I thought A is the weight matrix. In the example, where inputs are say situations and for each input we have the outcome as actions like pick, drop, move; what is being learnt so that the desired actions can be performed? Jul 30, 2014 at 5:56

We are learning a model to describe the relationship between inputs ($\mathbf{X}$) and output(s) ($\mathbf{y}$). What this means exactly depends on the nature of the model: it can be coefficients/weights, rules, ...

In supervised learning, we try to learn the relationship between inputs $\mathbf{X}$ and outputs $\mathbf{y}$. In general: $$\mathbf{y} = f(\mathbf{X}),$$ where $f(\cdot)$ is some unknown function that we want to learn (e.g. approximate its behaviour).

Consider a standard linear model, for which: $$\mathbf{y} \approx \mathbf{\hat{y}} = \mathbf{\beta}\mathbf{X}$$

The outputs $\mathbf{y}$ and inputs $\mathbf{X}$ are known. This is what we call training data. There is nothing to learn there. We try to learn how they are related to eachother as well as possible, which is the model (in this case $\mathbf{\beta}$).

Typically as well as possible means we want to optimize some kind of measure, for example maximizing the classification accuracy or minimizing the mean squared error of predictions $\mathbf{\hat{y}}$ compared to the truth $\mathbf{y}$.

In unsupervised learning we only try to describe the data at our disposal (e.g. $\mathbf{X}$). Clustering, for instance, attempts to retrieve sets of patterns that are similar to each other in some way.

• (A) for classification the output of the function we want to learn $f(\cdot)$ are class labels, (B) you try to build a model to learn the mapping $S\mapsto O$ and then use it to predict an optimal output for a new S. Jul 30, 2014 at 8:05