How Convolutional layer work exaclty in RGB image processing?

Question

I'm studying convolutional layers and I'm pretty confused. Supposing that I give to my network (CNN) an RGB image, so an image with three channels. Since the image has 3 channels, then the kernels applied to my image will be 3 in each convolutional layer (I don't care exactly about the size of the kernels at this moment).

Therefore the convolutional operation at the first layer will be given by: the sum of the products between each channel of the image and the corresponding channel of the kernel (see figure below)

However, if it works in this way, the output of the first convolutional layer would be an image of two dimensions and not an RGB image with 3 channels, as I think, it should be.

Since it wasn't clear, I have found the following picture hoping that it could help me. However, it wasn't clear as well. Could anyone help me?

Answer 1

However, if it works in this way, the output of the first convolutional layer would be an image of two dimensions and not an RGB image with 3 channels, as I think, it should be.

The output of the convolution between an image and a single kernel is a rank-2 tensor (has height and width, but only 1 channel).

A convolutional "filter" consists of one or more kernels, and the convolution between an image and a filter can be computed by taking all of the rank-2 outputs of convolving the image with each kernel, and stacking up to create a rank-3 tensor (has height, width, and "depth", or multiple channels).

---

Another way to describe things:

• An image is a CxHxW tensor (where C is usually 3)
• A kernel is a CxSxS tensor
• A filter is a KxCxSxS tensor
• Convolving an image with a kernel produces a H'xW' tensor (H',W' $\propto$ H,W).
• Convolving an image with a filter produces a KxH'xW' tensor