in a CNN, why isn't the "type" information obtained at one layer lost by the next layer?

Question

Here is something that bothers me about Convolutional Neural Networks, hopefully I can explain this well:

While the input to a given layer has channels, and the output FROM that layer has channels, the output for a given filter in that layer lacks channels. It is just a simple width*height image.

In a sense, we can say that - so far as that filter is concerned - the input channels have been compressed into something simpler.

But as a result, for any given number in the filter output, we can't tell if the number has a high value because of what the filter saw in (for example) an input channel representing vertical edges, or because of what the filter saw in an input channel representing horizontal edges, etc.

So it seems like we lose the "type" of information that was gained by the previous layer. Why isn't this a problem?

Answer 1

For classifying an object or animal, it's not necessary to remember the activations of the low-level neurons, which mostly learn to detect edges, corners, and gradients. So there's no problem with throwing away information from these neurons -- in fact the human vision system does the same thing -- no one interprets an object as a set of edges, you just see the object as a whole!

For tasks which require very precise information about each pixel, such as pixel-level segmentation (labeling what object every pixel of the image belongs to), deconvolutional layers and skip-connections are used to integrate both high-level semantic features from the upper layers along with the precision of the low-level features learned in the bottom layers.