# What's is the training data here?

Can somebody explain me why this classifier is giving a loss equal to cero? I don't get the example

SourceText

The basic setup from this problem is that we draw $$n$$ data points, where $$(x_i, y_i)$$ is the $$i$$th data point from our underlying distrbution $$\mathcal{D}$$. $$x_i$$ is the vector of features of the $$i$$th sample and $$y_i$$ is its associated label. The point of the passage is that minimizing the empirical risk seems to be an intuitive choice to create a "good" classifier. However, $$h_S$$ gives an example of where it can go wrong.
Indeed, $$h_S$$ is just memorizing the training data; i.e. if the input to $$h_S$$ is $$x_i$$, which is the vector of features of $$i$$th data point, then the output will be $$y_i$$ which is always the correct label. So, the empirical risk when using $$h_S$$ is always zero, as it outputs the correct label by design. However, it's not hard to see why this is not a good classifier; on unseen data, we always predict 0, without any regard for the data!
• I'll use a more canonical classification example. Let's say that we are trying to predict whether an email is spam ($y_i = 1$) or not ($y_i=0$) based off of the number of words in the email $x_i = \text {number of words in email } i$. Say I only have two data points $(92, 1)$ and $(133, 0)$. Then $h_S(92) = 1$ and $h_S(x) = 0$ for all $x \neq 92$. Dec 21, 2020 at 6:57
• $L_{\mathcal{D}}(h_S)$ is the "true" loss of $h_S$ not just on our training set. Read the section on area in the passage, it is very clear. Dec 21, 2020 at 7:02