This question already has an answer here:
I know that when dealing with artificial neural networks, RELU yields a value based on the weighted sum of the inputs plus a bias term. However, this logic does not seem to apply to convolutional neural networks.
Looking at the ResNet architecture, the outputs of the convolutional neural nets (what I believe to be feature maps), is added to the input x, and then RELU is applied onto it. What exactly does the RELU function do in this case? Do the convolution layers output feature maps, or something else?