Why is k called representer of evaluation in the definition of kernel functions

Question

Why is $k$ called representor of evaluation? From the book "Learning with kernels" by Bernhard Schölkopf we have the following lines (page 33):

$\langle k(.,x),f\rangle = f(x)$, in particular $\langle k(.,x), k(.,x')\rangle = k(x,x')$

According to the book this interesting property of $\phi$ follows from definition. How?

I am unable to understand this and this is crucial for understanding the concept of reproducing kernel Hilbert spaces. Any help appreciated.

Luckily, the part of the book which needs to be referred i.e. section 2.2.2 (starts on page 32) is a part of the preview in Google books. Also note that this section is independent of other sections.

Answer 1

When it says "follows directly from the definition", it means directly from the definition of $\langle -,- \rangle$, not directly from the definition of $\Phi$. Equation 2.24 on page 33 is a definition (that's why it uses the := notation instead of just an equals sign.) The definition says that if $f = \sum_i \alpha_i k(.,x_i)$ and $g = \sum_j \beta_j k(.,y_j)$ then $\langle f,g\rangle$ is defined to be $\sum_{i,j} \alpha_i \beta_j k(x_i, y_j)$. In particular, if $f=k(.,x)$ and $g=k(.,x')$ then the definiton in Equation 2.24 says that $$\langle f, g \rangle = k(x,x')$$ (think of one $\alpha_i$ and one $\beta_j$ being $1$ and the rest $0$). This is Equation 2.30.

Equation 2.29 is similar but more general. Suppose we have some $f = \sum_j \beta_j k(.,x_j)$. Then $$\langle k(,.x), f \rangle = \langle k(.,x), \sum\beta_j k(.,x_j) \rangle$$ and again Definition 2.24 says that this equals $$\sum \beta_j k(x, x_j)$$ by definition. But this is just $f(x)$, so that gives (2.29).

Answer 2

In an RKHS framework, any function $f(\cdot)$ can be minimized with a hilbert-norm minimizing solution, just by a linear combination of the kernels evaluated at the rest of the data points and $x$ itself.

i.e, in detail, if you dont know the result of the function $f's$ Hilbert-norm minimizing minimizer, all you need to do in a reproducing kernel Hilbert space, (RKHS) is that you take the reproducing p.s.d kernel $K(\cdot,\cdot)$ which is a function of two arguments (pairwise), where in one argument you fix $x$; the point where you would like to evaluate $f(\cdot)$ such that $x$ is the minimizer and then in the other argument you choose each of the other points apart from $x$ that you have in your dataset, as $K(x,\cdot)$ and then the "Representer Theorem" ensures that there exists a set of $\alpha's$ which go along with $K(x,\cdot)$ such that the result $f(x)$ "can be" represented/${evaluated}/admits$ the following property: $f(x)=\sum{K(x,\cdot)}$ where the placeholder $\cdot$ takes the rest of the data points and the summation is over each of those points.