26
$\begingroup$

When people talk about neural networks, what do they mean when they say "kernel size"? Kernels are similarity functions, but what does that say about kernel size?

$\endgroup$
1
  • 2
    $\begingroup$ In a CNN context, people sometimes use "kernel size" to mean the size of a convolutional filter, and likewise a "kernel" is the filter itself. $\endgroup$ – Sycorax Aug 7 '17 at 18:44
30
$\begingroup$

Deep neural networks, more concretely convolutional neural networks (CNN), are basically a stack of layers which are defined by the action of a number of filters on the input. Those filters are usually called kernels.

For example, the kernels in the convolutional layer, are the convolutional filters. Actually no convolution is performed, but a cross-correlation. The kernel size here refers to the widthxheight of the filter mask.

The max pooling layer, for example, returns the pixel with maximum value from a set of pixels within a mask (kernel). That kernel is swept across the input, subsampling it.

So nothing to do with the concept of kernels in support vector machines or regularization networks. You can think of them as feature extractors.

$\endgroup$
31
$\begingroup$

enter image description here

As you can see above, the kernel, also known as kernel matrix is the function in between and its size, here 3, is the kernel size(where kernel width equals kernel hight).

Note that the kernel does not necessarily to be symmetric, and we can verify that by quoting this text from the doc of Conv2D in Tensorflow:

kernel_size: An integer or tuple/list of 2 integers, specifying the height and width of the 2D convolution window. Can be a single integer to specify the same value for all spatial dimensions.

But usually, we just make the width and height equal, and if not the kernel size should be a tuple of 2. The kernel can be unsymmetric for instance in Conv1D(see this example, and the kernel size can be more than 2 numbers, for example (4, 4, 3) in the example bellow Conv3D:

enter image description here

The awesome gifs come from here and here.

$\endgroup$
3
  • $\begingroup$ Is it necessary that the kernel size has to be symmetric? $\endgroup$ – Ben Mar 12 '20 at 6:15
  • 1
    $\begingroup$ @Ben No, it's unnecessary, so the kernel size can be a tuple of 1, 2 or 3 numbers. $\endgroup$ – Lerner Zhang Mar 12 '20 at 15:14
  • $\begingroup$ This is a great explanation and the visuals are very helpful. Thank you! $\endgroup$ – Abhishek Arora Sep 13 '20 at 20:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.