Skip to main content
Cross Validated

Return to Question

added 2 characters in body
Source Link
Jack M
Jack M
  • 439
  • 4
  • 8

Suppose I have a series of labelled training inputs $(x_i, y_i)$, and a kernel function $k$ on the input domain, with a corresponding RKHS $H$. Now form the Gram matrix $A$, where $A_{ij}=k(x_i, x_j)$. If $A$ is positive-definite, then there is a unique solution to

$$A\alpha=y$$

where $y$ is the vector of training labels. If I solve that system, then the function

$$f(x)=\sum_i\alpha_ik(x,x_i)$$

is, if I understand correctly, the unique functionminimum RKHS norm $f$ in the RKHS $H$ such that $f(x_i)=y_i$ for all $i$, by the Representer Theorem.

This defines a supervised learning procedure, by which I mean an algorithm which given a labelled training set, produces a hypothesis function on the input domain. Does this learning procedure have a name? I don't think it's equivalent to a Support Vector Machine, which is usually what comes up when I try googling for something like this. Kernel method seems to mean something more general, and kernel regression seems to mean something different - I don't recognize any of the formulae in that article.

Suppose I have a series of labelled training inputs $(x_i, y_i)$, and a kernel function $k$ on the input domain, with a corresponding RKHS $H$. Now form the Gram matrix $A$, where $A_{ij}=k(x_i, x_j)$. If $A$ is positive-definite, then there is a unique solution to

$$A\alpha=y$$

where $y$ is the vector of training labels. If I solve that system, then the function

$$f(x)=\sum_i\alpha_ik(x,x_i)$$

is, if I understand correctly, the unique function $f$ in the RKHS $H$ such that $f(x_i)=y_i$ for all $i$, by the Representer Theorem.

This defines a supervised learning procedure, by which I mean an algorithm which given a labelled training set, produces a hypothesis function on the input domain. Does this learning procedure have a name? I don't think it's equivalent to a Support Vector Machine, which is usually what comes up when I try googling for something like this. Kernel method seems to mean something more general, and kernel regression seems to mean something different - I don't recognize any of the formulae in that article.

Suppose I have a series of labelled training inputs $(x_i, y_i)$, and a kernel function $k$ on the input domain, with a corresponding RKHS $H$. Now form the Gram matrix $A$, where $A_{ij}=k(x_i, x_j)$. If $A$ is positive-definite, then there is a unique solution to

$$A\alpha=y$$

where $y$ is the vector of training labels. If I solve that system, then the function

$$f(x)=\sum_i\alpha_ik(x,x_i)$$

is, if I understand correctly, the minimum RKHS norm $f$ in the RKHS $H$ such that $f(x_i)=y_i$ for all $i$, by the Representer Theorem.

This defines a supervised learning procedure, by which I mean an algorithm which given a labelled training set, produces a hypothesis function on the input domain. Does this learning procedure have a name? I don't think it's equivalent to a Support Vector Machine, which is usually what comes up when I try googling for something like this. Kernel method seems to mean something more general, and kernel regression seems to mean something different - I don't recognize any of the formulae in that article.

Source Link
Jack M
Jack M
  • 439
  • 4
  • 8

What is it called when you find the best fit in an RKHS to some training data?

Suppose I have a series of labelled training inputs $(x_i, y_i)$, and a kernel function $k$ on the input domain, with a corresponding RKHS $H$. Now form the Gram matrix $A$, where $A_{ij}=k(x_i, x_j)$. If $A$ is positive-definite, then there is a unique solution to

$$A\alpha=y$$

where $y$ is the vector of training labels. If I solve that system, then the function

$$f(x)=\sum_i\alpha_ik(x,x_i)$$

is, if I understand correctly, the unique function $f$ in the RKHS $H$ such that $f(x_i)=y_i$ for all $i$, by the Representer Theorem.

This defines a supervised learning procedure, by which I mean an algorithm which given a labelled training set, produces a hypothesis function on the input domain. Does this learning procedure have a name? I don't think it's equivalent to a Support Vector Machine, which is usually what comes up when I try googling for something like this. Kernel method seems to mean something more general, and kernel regression seems to mean something different - I don't recognize any of the formulae in that article.