Applying the "kernel trick" to linear methods? The kernel trick is used in several machine learning models (e.g. SVM).  It was first introduced in the "Theoretical foundations of the potential function method in pattern recognition learning" paper in 1964.  
The wikipedia definition says that it is 

a method for using a linear classifier
  algorithm to solve a non-linear
  problem by mapping the original
  non-linear observations into a
  higher-dimensional space, where the
  linear classifier is subsequently
  used; this makes a linear
  classification in the new space
  equivalent to non-linear
  classification in the original space.

One example of a linear model that has been extended to non-linear problems is the kernel PCA.  Can the kernel trick be applied to any linear model, or does it have certain restrictions?
 A: Two further references from B. Schölkopf:


*

*Schölkopf, B. and Smola, A.J. (2002). Learning with kernels. The MIT Press.

*Schölkopf, B., Tsuda, K., and Vert, J.-P. (2004). Kernel methods in computational biology. The MIT Press.


and a website dedicated to kernel machines.
A: @ebony1 gives the key point (+1), I was a co-author of a paper discussing how to kernelize generalised linear models, e.g. logistic regression and Poisson regression, it is pretty straightforward.
G. C. Cawley, G. J. Janacek and N. L. C. Talbot, Generalised kernel machines, in Proceedings of the IEEE/INNS International Joint Conference on Neural Networks (IJCNN-2007), pages 1732-1737, Orlando, Florida, USA, August 12-17, 2007. (www,pdf)
I also wrote a (research quality) MATLAB toolbox (sadly no instructions), which you can find here.
Being able to model the target distribution is pretty useful in uncertainty quantification etc. so it is a useful (if fairly incremental) addition to kernel learning methods.
A: The kernel trick can only be applied to linear models where the examples in the problem formulation appear as dot products (Support Vector Machines, PCA, etc).
