Kernel transformation in Machine Learning

Question

I understand kernels allow us to linearly separate non-linearly separable data in a higher-dimensional space.

Given a feature vector $\bar x = [x1,x2,..xn]^T$, we can apply the transformation $\phi(\bar x)$, and apply the usual regression $ y = \bar w^T\phi(\bar x)$.

However, I do not understand the notation in the following question:

Given N data points $(x,t)$ (scalars), fit an 𝑀th degree polynomial using polynomial and Gaussian kernels, and study goodness of fit.

---

To be more specific, what function $\phi$ do I use in the Polynomial and Gaussian kernels to obtain the transformed input vector?

Answer 1

I was confused whether we use the explicit representation as in $\phi(\bar x)$ or do it implicitly using the Gram matrix $K$; $K(i,j)$ = $<\phi(\bar x_j),\phi(\bar x_i)>$. To have $\phi$ transform data into an $infinite$-dimensional space for the Gaussian, we use the implicit representation, and then we can proceed to use the least-squares solution to compute the optimum $\bar \alpha$ estimate.

$\bar \alpha$ = $ (K + \lambda I)^{-1}Y$, where $\lambda$ is the regularisation parameter and $Y$ is the target variable.

The estimate $\hat Y(i)$ = $\bar \alpha'k(\bar x_j,\bar x_j)$

For the polynomial kernel of order d: $K(i,j)$= $\ <\phi(\bar x_j),\phi(\bar x_i)>\ =(\bar x_j'\bar x_i)^d $

For the Gaussian kernel of variance $\sigma ^2$: $K(i,j) =\ <\phi(\bar x_j),\phi(\bar x_i)>\ = e^{(||\bar x_j-\bar x_i||)/2\sigma^2}/\sqrt(2\pi \sigma ^2) $