What's a "patch" in CNN? By reading around, a "patch" seems to be a subsection of an input image to the CNN, but what exactly is it? When does a "patch" come into play when solving problems using CNN? Why do we need "patches"? What's the relation between a "patch" and a kernel (i.e. the feature detector)?
I would appreciate if you can systematically answer to all those questions.
 A: 
By reading around, a "patch" seems to be a subsection of an input image to the CNN, but what exactly is it?

It's exactly what you describe. The kernel (or filter or feature detector) only looks at one chunk of an image at a time, then the filter moves to another patch of the image, and so on.

When does a "patch" come into play when solving problems using CNN? 

When you apply a CNN filter to an image, it looks at one patch at a time.

Why do we need "patches"?

CNN kernels/filters only process one patch at a time, rather than the whole image. This is because we want filters to process small pieces of the image in order to detect features (edges, etc). This also has a nice regularization property, since we're estimating a smaller number of parameters, and those parameters have to be "good" across many regions of each image, as well as many regions of all other training images.

What's the relation between a "patch" and a kernel (i.e. the feature detector)?

The patch is the input to the kernel.
