Kernel functions are used commonly with SVMs to make classification of non linearly separable data possible - i.e. the Kernel function provides the linear separability. But from looking at Kernel functions it seems to me that they produce a scalar output....
If I just want to make data linearly separable (and not immediately use some classifier on it), how do I make any given Kernel function (for example, an RBF kernel), produce a vector output?...
many thanks in advance.
It sounds like what you actually want is to map points into the feature space defined by the kernel function.
Kernel functions produce scalar outputs by definition. Given inputs $x_1, x_2$, a kernel function $k$ outputs the inner product (i.e. dot product) of $x_1$ and $x_2$ when mapped into a feature space:
$$k(x_1, x_2) = \langle \Phi(x_1), \Phi(x_2) \rangle$$
Here, $\Phi$ represents a (possibly nonlinear) mapping from input space to feature space, and is defined implicitly by the kernel function. Kernel methods (e.g. SVMs) don't need to compute this mapping explicitly because they can work directly with the inner products.
But, you want the feature space representations themselves. Suppose you have data points $\{x_1, \dots, x_n\}$ and compute kernel matrix $K$ where $K_{ij} = k(x_i, x_j)$. Let matrix $Y$ denote the feature space representations of the data, where the $i$th row contains $\phi(x_i)$. Then, according to the equation above, $Y$ is any solution to the equation:
$$Y Y^T = K$$
Note that the solution is only specified up to rotations and sign flips, as these operations don't change the dot product. Also, feature space may be very high dimensional in some cases (or even infinite dimensional, as for the RBF kernel). However, the feature space representations recovered here will have dimensionality at most $n$, because $n$ data points can't span a higher dimensional subspace.
In practice, you can solve this equation using an eigendecomposition of the kernel matrix. Or, similarly, use kernel PCA, which is implemented in many software packages. This is a kernelized version of PCA that's equivalent to ordinary PCA operating in feature space. If desired, you can discard the dimensions with lowest variance to obtain approximate feature space representations that are lower dimensional.
