Output shape of a convolutional filter Below I have the following architecture. I have input image $28\times 28$ and the following convolutional and pooling layers:
$28\times 28 -> 32 (3\times 3) kernels -> (2\times 2) pooling -> 64 (3\times 3) kernels-> (2\times 2) pooling$
To my understanding:
First output: after applying $32 (3\times 3) kernels$ is $28-3 + 1 = 26$ (for both width and height), so we will have $32 (26 \times 26)$ output.
Second output: after applying $(2\times 2) pooling$ on the first output is $o=\lfloor \frac{26}{2} \rfloor =13$ (for both width and height), so we will have $32 (13 \times 13)$ output.
Third output: after applying $64 (3\times 3) kernels$ on the second output (for both width and height), so we will have $64 (11 \times 11)$ output. But I am not sure why we have $64 (11 \times 11)$ output given that we apply $64 (11 \times 11)$ on each $32 (13 \times 13)$ output, so we should have $64 \times 32 (11 \times 11)$, please?
 A: Convolution operation in neural networks also has a channel dimension. The kernels will always have a channel dimension equal to the input channel dimension. The input and weight channel dimension are multiplied and accumulated as well, thus disappear.
First layer: 28 x 28 input image with only 1 channel is filtered with 32 3 x 3 kernels. 32 is also referred as output feature maps (i.e. output channels). So, the output will be 26 x 26 image as you said, but with 32 channels.
Second layer: this is pooling, and as you mentioned, the output 13 x 13 images with 32 channels.
Third layer: There are 64 3 x 3 kernels, so the number of output channels is 64. It's not stated, but the kernel also has a hidden channel dimension which must be equal to the input channel dimension.  Thus each kernel is of 3 x 3 x 32 shape (channels last notation). The channel dimension is also accumulated while sliding the kernel, so it disappears. There are 64 kernels, then, the output of this layer is 11 x 11 images with 64 channels.
