# What does a kernel function do in English

The kernel trick avoids the explicit mapping that is needed to get linear learning algorithms to learn a nonlinear function or decision boundary. For all and in the input space , certain functions can be expressed as an inner product in another space . The function is often referred to as a kernel or a kernel function.

To be quite honest, I have no idea what this means. But this is what I think it means:

Kernel function is a pre-processing/reducing/refining step that takes vector representations of objects into it, and spits out an easier-to-deal-with representation of it - that we can then better use in general learning algorithms

For example (following with this analogy):

If my task was to develop a "learning algorithm" to rank people by height based on photos. A good "kernel function" would be one that takes all the photos in and draws red lines from their heads to their toes (indicating their height) - So now the learning algorithm no longer needs to do all this fancy geometrical acrobatics to extract the height, but merely measure the distance of these long lines (making life simpler).

Is this the gist of what kernel functions do? (act as mathematical vector pre-processors)? I don't really understand the whole deal with:

certain functions can be expressed as an inner product in another space

Example: say, you have a data series $IQ_i$ and $IN_i$ of intellect and income, where $i=1,\dots,n$ are student ids. Say we got a new student $i=n+1$, and we only know how much money $IN_{n+1}$ he has. We want to predict his intellect. We could use a kernel function $K_{ij}=K(IN_i,IN_j)=\kappa(|IN_i-IN_j|)$ as follows: $$IQ_{n+1}=\sum_{i=1}^n\kappa(IN_i-IN_{n+1})\cdot IQ_i=\sum_{i=1}^nK_{n+1,i} IQ_i\equiv K\cdot IQ$$
• Can you explain exactly what you mean with your long equation? From what I am getting from it is pretend this new guy's IQ is the (average?) difference between all the incomes of our dataset??? – AlanSTACK Mar 4 '16 at 17:21
• @Alan, For instance, you could build a nonlinear model $IQ=f(IN|\theta)+\varepsilon$, then estimate its parameters $\theta$. The trouble is to come up with an appropriate function $f(.)$. Instead, with the kernel you avoid this alltogether – Aksakal Mar 4 '16 at 18:50