Intro Background

Within a convolutional neural network, we usually have a general structure / flow that looks like this:

  1. input image (i.e. a 2D vector x)

(1st Convolutional layer (Conv1) starts here...)

  1. convolve a set of filters (w1) along the 2D image (i.e. do the z1 = w1*x + b1 dot product multiplications), where z1 is 3D, and b1 is biases.
  2. apply an activation function (e.g. ReLu) to make z1 non-linear (e.g. a1 = ReLu(z1)), where a1 is 3D.

(2nd Convolutional layer (Conv2) starts here...)

  1. convolve a set of filters along the newly computed activations (i.e. do the z2 = w2*a1 + b2 dot product multiplications), where z2 is 3D, and and b2 is biases.
  2. apply an activation function (e.g. ReLu) to make z2 non-linear (e.g. a2 = ReLu(z2)), where a2 is 3D.

 The Question

The definition of the term "feature map" seems to vary from literature to literature. Concretely:

  • For the 1st convolutional layer, does "feature map" corresponds to the input vector x, or the output dot product z1, or the output activations a1, or the "process" converting x to a1, or something else?
  • Similarly, for the 2nd convolutional layer, does "feature map" corresponds to the input activations a1, or the output dot product z2, or the output activation a2, or the "process" converting a1 to a2, or something else?

In addition, is it true that the term "feature map" is exactly the same as "activation map"? (or do they actually mean two different thing?)

 Additional references:

Snippets from Neural Networks and Deep Learning - Chapter 6:

*The nomenclature is being used loosely here. In particular, I'm using "feature map" to mean not the function computed by the convolutional layer, but rather the activation of the hidden neurons output from the layer. This kind of mild abuse of nomenclature is pretty common in the research literature.

Snippets from Visualizing and Understanding Convolutional Networks by Matt Zeiler:

In this paper we introduce a visualization technique that reveals the input stimuli that excite individual feature maps at any layer in the model. [...] Our approach, by contrast, provides a non-parametric view of invariance, showing which patterns from the training set activate the feature map. [...] a local contrast operation that normalizes the responses across feature maps. [...] To examine a given convnet activation, we set all other activations in the layer to zero and pass the feature maps as input to the attached deconvnet layer. [...] The convnet uses relu non-linearities, which rectify the feature maps thus ensuring the feature maps are always positive. [...] The convnet uses learned filters to convolve the feature maps from the previous layer. [...] Fig. 6, these visualizations are accurate representations of the input pattern that stimulates the given feature map in the model [...] when the parts of the original input image corresponding to the pattern are occluded, we see a distinct drop in activity within the feature map. [...]

Remarks: also introduces the term "feature map" and "rectified feature map" in Fig 1.

Snippets from Stanford CS231n Chapter on CNN:

[...] One dangerous pitfall that can be easily noticed with this visualization is that some activation maps may be all zero for many different inputs, which can indicate dead filters, and can be a symptom of high learning rates [...] Typical-looking activations on the first CONV layer (left), and the 5th CONV layer (right) of a trained AlexNet looking at a picture of a cat. Every box shows an activation map corresponding to some filter. Notice that the activations are sparse (most values are zero, in this visualization shown in black) and mostly local.

Snippets from A-Beginner's-Guide-To-Understanding-Convolutional-Neural-Networks

[...] Every unique location on the input volume produces a number. After sliding the filter over all the locations, you will find out that what you’re left with is a 28 x 28 x 1 array of numbers, which we call an activation map or feature map.

  • $\begingroup$ The nomenclature seems misleading since map means mapping from one thing to another, but in this scenario the feature map is a result of the convolution and the filters are more like maps, mapping the input to the output/next layer. To remember that, I just think of the feature map as a place to check where a particular feature/pattern lies. $\endgroup$ Commented Oct 31, 2022 at 14:36

4 Answers 4


A feature map, or activation map, is the output activations for a given filter (a1 in your case) and the definition is the same regardless of what layer you are on.

Feature map and activation map mean exactly the same thing. It is called an activation map because it is a mapping that corresponds to the activation of different parts of the image, and also a feature map because it is also a mapping of where a certain kind of feature is found in the image. A high activation means a certain feature was found.

A "rectified feature map" is just a feature map that was created using Relu. You could possibly see the term "feature map" used for the result of the dot products (z1) because this is also really a map of where certain features are in the image, but that is not common to see.

  • 1
    $\begingroup$ Thanks for the input. Your response aligns with my understanding (i.e. activation maps are the a1, a2 etc). In Conv2, I guess I would call a1 the input activation map, and a2 the output activation map. In Conv1, I x the input image, and a1 the output activation map. $\endgroup$
    – Atlas7
    Commented Jul 20, 2017 at 10:39
  • $\begingroup$ "A mapping of where a certain kind of feature is found in the image". Can we say an approximate mapping since a feature map can be smaller than the original image? For example, suppose that we have a feature map very deep in a CNN that detect faces and has size $2 \times 2$. If the activation happens only at the first row and column we can say that a face was detected at the upper quadrant of the image but we don't exactly where. $\endgroup$
    – ado sar
    Commented Jul 28, 2023 at 9:37

enter image description here

In CNN terminology, the 3×3 matrix is called a ‘filter‘ or ‘kernel’ or ‘feature detector’ and the matrix formed by sliding the filter over the image and computing the dot product is called the ‘Convolved Feature’ or ‘Activation Map’ or the ‘Feature Map‘. It is important to note that filters acts as feature detectors from the original input image.

source : https://ujjwalkarn.me/2016/08/11/intuitive-explanation-convnets/


before talk about what feature map means, let just define the term of feature vector.

feature vector is vectorial representation of objects. For example, a car can be represented by [number of wheels, door. windows, age ..etc].

feature map is a function that takes feature vectors in one space and transforms them into feature vectors in another. For example given a feature vector [volume ,weight, height, width] it can return [1, volume/weight, height * width] or [height * width] or even just [volume]


To give a complete answer, we need some definitions:

Background Definitions:

  • For us, an "input space" $\mathcal{X}$ is just a metric space.
  • A model class $\mathcal{F}$ (of continuous functions) is universal from $\mathcal{X}$ to $\mathcal{R}^D$ if $\mathcal{F}$ is dense in $C(\mathcal{X},\mathbb{R}^D)$ for the uniform convergence on compacts topology.

Definition of a Feature Map:

A feature map implicitly depends on the learning model class used and on the "input space" $\mathcal{X}$ where the data lies. More formally, if $\mathcal{F}$ is a class of models from $\mathbb{R}^d$ to $\mathbb{R}^D$ then a feature map for $\mathcal{F}$ on an input space $\mathcal{X}$ is a (just) function $$ \phi:\mathcal{X}\rightarrow \mathbb{R}^d . $$

What's the point of a feature map?:

  1. The (first) point here is that $\phi$ makes the data in $\mathcal{X}$ compatable with the learning model in $F$; i.e.: $\mathcal{F}_{\phi}\triangleq \{\hat{f}\circ \phi:\, f\in \mathcal{F}\}$ is not a set of models from $\mathcal{X}$ to $\mathbb{R}^D$.

What is a "good" feature map?

  1. The (second) point is that a "good" choice of a feature map (even in the case where $\mathcal{X}=\mathbb{R}^d$) can strictly improve the expressibility of the model class $\mathcal{F}$. This means that:

a. (Upgrading Property) $\mathcal{F}$ is "universal" then so is $\mathcal{F}_{\phi}$

b. (UAP-Invariance Property) if $\mathcal{F}$ is "universal" then so is $\mathcal{F}_{\phi}$.

Literature Review on Constructing "Good Feature Maps:


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.