7
$\begingroup$

When reading about the transforming the fully connected layer into convolutional layer, posted in http://cs231n.github.io/convolutional-networks/#convert.

I just feel confused about the following two comments:

It turns out that this conversion allows us to "slide" the original ConvNet very efficiently across many spatial positions in a larger image, in a single forward pass.

A standard ConvNet should be able to work on any size image. The convolutional filter can slide across the image grid, so why do we need to slide the original ConvNet in any spatial positions in a larger image?

And

Evaluating the original ConvNet (with FC layers) independently across 224x224 crops of the 384x384 image in strides of 32 pixels gives an identical result to forwarding the converted ConvNet one time.

What does "strides of 32 pixels" mean here? Does that refer to the filter size? When talking about 224*224 crops of the 384*384 image, does that mean we use a receptive field of 224*224?

I marked these two comments as red in the original context.

enter image description here

$\endgroup$
1
  • 3
    $\begingroup$ You need to read up on ConvNet. Stride means jump size. "When the stride is 1 then we move the filters one pixel at a time. When the stride is 2 (or uncommonly 3 or more, though this is rare in practice) then the filters jump 2 pixels at a time as we slide them around. This will produce smaller output volumes spatially." source : cs231n.github.io/convolutional-networks $\endgroup$
    – horaceT
    Jun 30 '16 at 22:38
7
$\begingroup$

Fully connected layers can only deal with input of a fixed size, because it requires a certain amount of parameters to "fully connect" the input and output. While convolutional layers just "slide" the same filters across the input, so it can basically deal with input of an arbitrary spatial size.

In the example network with fully-connected layers at the end, a 224*224 image will output an 1000d vector of class scores. If we apply the network on a larger image, the network will fail because of the inconsistency between the input and parameters of the first fully-connected layer.

One the other hand, if we use a fully convolutional network, when applied to a larger image we'll get 1000 "heatmaps" of class scores.

As shown in the following figure (from the FCN segmentation paper), the upper network gives one score per class, and after the conversion (convolutionalization), we can get the a heatmap per class for a larger image.
enter image description here

About "stride", on the same page, in the section Spatial arrangement:

When the stride is 1 then we move the filters one pixel at a time. When the stride is 2 (or uncommonly 3 or more, though this is rare in practice) then the filters jump 2 pixels at a time as we slide them around. This will produce smaller output volumes spatially.

$\endgroup$
15
  • 1
    $\begingroup$ @user3269 if the input of the fully-connect layer is 7*7*512, we'll need 7*7*512*4096 parameters to "fully connect" the input and the output. if the spatial size of the input increases, we'll need more parameters. for convolutional layers we just apply the same filter to every part of the input. $\endgroup$
    – dontloo
    Jul 2 '16 at 7:56
  • 2
    $\begingroup$ @user3269 the network output one value per class for 224*224 images, when we feed in a 384*384 image, it will output a 12*12 heatmap for each class, each pixel of the heatmap is equivalent to the value we get by applying the network on a 224*224 patch in a specific location. because the network reduces the spatial size of he input by a factor of 32 (224*224->7*7), so sliding by one pixel in the top layer is sliding by 32 pixels in the input image. $\endgroup$
    – dontloo
    Jul 2 '16 at 8:08
  • 1
    $\begingroup$ what if we "slide" on some larger input size with a fc cnn, we could still get a "heatmap". If so, what's the point of a fully conv network? $\endgroup$
    – flankechen
    Dec 5 '18 at 13:04
  • 1
    $\begingroup$ @flankechen hi, if I understood correctly, sliding an entire cnn to get a heatmap is less efficient since there will be large overlaps among different windows sent into the network, and we will end up applying the same set of filters to these overlapping regions repeatedly. A better choice is to save such redundancy and apply the network to the entire image once, and that can be done by a fully convolutional network. $\endgroup$
    – dontloo
    Dec 5 '18 at 13:41
  • 1
    $\begingroup$ @flankechen for fully conv networks send the entire image into the network and slide the convolution filters which is more efficient than sliding the whole network, apart from that I think a fully connected network would work as well. $\endgroup$
    – dontloo
    Dec 6 '18 at 6:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.