There are several distinct usages:

  • kernel density estimation
  • kernel trick
  • kernel smoothing

Please explain what the "kernel" in them means, in plain English, in your own words.

  • 3
    $\begingroup$ Not to be rude, but isn't this a question that is already answered ad nausea on Wikipedia and the likes? Google gave me the answer within 15 seconds... $\endgroup$
    – Joris Meys
    Commented Sep 9, 2010 at 8:32
  • 54
    $\begingroup$ I absolutely hate wikipedia answers for stats. There are rambling, symbolic messes. I am looking for a gem of an answer that can explain the answer in plain English, as I believe that that shows a deeper level of understanding than a math equation. There are many popular "plain English" questions on here, and for good reason. $\endgroup$ Commented Sep 9, 2010 at 18:04

2 Answers 2


In both statistics (kernel density estimation or kernel smoothing) and machine learning (kernel methods) literature, kernel is used as a measure of similarity. In particular, the kernel function $k(x,.)$ defines the distribution of similarities of points around a given point $x$. $k(x,y)$ denotes the similarity of point $x$ with another given point $y$.

  • $\begingroup$ This is a nice way of putting it. I am wondering if you can generalize this description to also apply to the kernel of 'kernel density estimation'. $\endgroup$
    – shabbychef
    Commented Sep 9, 2010 at 16:18
  • 2
    $\begingroup$ In a way, yes. One way to understand kernel density estimation is that you approximate the density of a point from some distribution as a weighted average of its similarities with a set of points from the distribution. So the notion of similarity does play a role here as well. $\endgroup$
    – ebony1
    Commented Sep 9, 2010 at 17:14
  • 1
    $\begingroup$ I understand "kernel" in statistics to be borrowed originally from jargon used in discussion of integral equations. $\endgroup$
    – Nick Cox
    Commented Sep 16, 2014 at 14:47

There appear to be at least two different meanings of "kernel": one more commonly used in statistics; the other in machine learning.

In statistics "kernel" is most commonly used to refer to kernel density estimation and kernel smoothing.

A straightforward explanation of kernels in density estimation can be found (here).

In machine learning "kernel" is usually used to refer to the kernel trick, a method of using a linear classifier to solve a non-linear problem "by mapping the original non-linear observations into a higher-dimensional space".

A simple visualisation might be to imagine that all of class $0$ are within radius $r$ of the origin in an x, y plane (class $0$: $x^2 + y^2 < r^2$); and all of class $1$ are beyond radius $r$ in that plane (class $1$: $x^2 + y^2 > r^2$). No linear separator is possible, but clearly a circle of radius $r$ will perfectly separate the data. We can transform the data into three dimensional space by calculating three new variables $x^2$, $y^2$ and $\sqrt{2}xy$. The two classes will now be separable by a plane in this 3 dimensional space. The equation of that optimally separating hyperplane where $z_1 = x^2, z_2 = y^2$ and $z_3 = \sqrt{2}xy$ is $z_1 + z_2 = 1$, and in this case omits $z_3$. (If the circle is off-set from the origin, the optimal separating hyperplane will vary in $z_3$ as well.) The kernel is the mapping function which calculates the value of the 2-dimensional data in 3-dimensional space.

In mathematics, there are other uses of "kernels", but these seem to be the main ones in statistics.

  • 2
    $\begingroup$ Very nice! I'm going to use your example with the circle to explain kernel methods, as it is the best visualization I met up til now. Thanks! $\endgroup$
    – Joris Meys
    Commented Sep 10, 2010 at 13:47
  • 1
    $\begingroup$ Following up Thylacoleo's example using the circle to explain the kernel trick (i don't have enough reputation to add a comment directly to his answer) Was there a simple typo in the equation for the separating hyperplane? and it should be z1 + z2 = r^2, instead of z1 + z2 = 1? Or do I misunderstand? I agree its a nice simple example to illustrate the concept. Thanks. Though the definition of z3 still seems a bit of a mystery, but apparently it doesn't matter for the example centered at the origin. $\endgroup$ Commented Sep 11, 2010 at 1:37
  • $\begingroup$ Yes there was a typo. Thanks for that Alex. I don't always proofread :-) $\endgroup$
    – Thylacoleo
    Commented Sep 13, 2010 at 7:26
  • 2
    $\begingroup$ The following video was proposed by an anonymous potential editor as "a great visualisation of what Thylacoleo explained:" youtube.com/watch?v=3liCbRZPrZA $\endgroup$ Commented Jun 6, 2013 at 22:56
  • $\begingroup$ Do we use inner products t o map 2-dimensional data to 3-dimensional? $\endgroup$
    – SmallChess
    Commented Nov 9, 2015 at 3:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.