What is it called when you find the best fit in an RKHS to some training data?

Question

Suppose I have a series of labelled training inputs $(x_i, y_i)$, and a kernel function $k$ on the input domain, with a corresponding RKHS $H$. Now form the Gram matrix $A$, where $A_{ij}=k(x_i, x_j)$. If $A$ is positive-definite, then there is a unique solution to

$$A\alpha=y$$

where $y$ is the vector of training labels. If I solve that system, then the function

$$f(x)=\sum_i\alpha_ik(x,x_i)$$

is, if I understand correctly, the minimum RKHS norm $f$ in the RKHS $H$ such that $f(x_i)=y_i$ for all $i$, by the Representer Theorem.

This defines a supervised learning procedure, by which I mean an algorithm which given a labelled training set, produces a hypothesis function on the input domain. Does this learning procedure have a name? I don't think it's equivalent to a Support Vector Machine, which is usually what comes up when I try googling for something like this. Kernel method seems to mean something more general, and kernel regression seems to mean something different - I don't recognize any of the formulae in that article.

Answer 1

It's kernel ridge regression with zero weight decay (zero regularisation, $\lambda=0$).

Denote by $\textbf{k}$ the vector whose components are $k_i = k(x, x_i)$. Then you can write your $f(\textbf{x})$ in a vector form:

$$ f(\textbf{x}) = \alpha^T \textbf{k} = \textbf{y}^T \textbf{A}^{-1} \textbf{k} $$

For comparison, kernel ridge regression is given by

$$ f(\textbf{x}) = \textbf{y}^T (\textbf{A} + \lambda \textbf{I}) ^{-1} \textbf{k} $$

For reference, see e.g. Slide 31 here, or p. 119 in Cristianini and Shawe-Taylor, "An Introduction to Support Vector Machines and other kernel-based machine learning methods".