7
$\begingroup$

When reading about the transforming the fully connected layer into convolutional layer, posted in http://cs231n.github.io/convolutional-networks/#convert.

I just feel confused about the following two comments:

It turns out that this conversion allows us to "slide" the original ConvNet very efficiently across many spatial positions in a larger image, in a single forward pass.

A standard ConvNet should be able to work on any size image. The convolutional filter can slide across the image grid, so why do we need to slide the original ConvNet in any spatial positions in a larger image?

And

Evaluating the original ConvNet (with FC layers) independently across 224x224 crops of the 384x384 image in strides of 32 pixels gives an identical result to forwarding the converted ConvNet one time.

What does "strides of 32 pixels" mean here? Does that refer to the filter size? When talking about 224*224 crops of the 384*384 image, does that mean we use a receptive field of 224*224?

I marked these two comments as red in the original context.

enter image description here

Improve this question
$\endgroup$
1
  • 3
    $\begingroup$ You need to read up on ConvNet. Stride means jump size. "When the stride is 1 then we move the filters one pixel at a time. When the stride is 2 (or uncommonly 3 or more, though this is rare in practice) then the filters jump 2 pixels at a time as we slide them around. This will produce smaller output volumes spatially." source : cs231n.github.io/convolutional-networks $\endgroup$
    – horaceT
    Commented Jun 30, 2016 at 22:38

1 Answer 1

Reset to default
8
$\begingroup$

Fully connected layers can only deal with input of a fixed size, because it requires a certain amount of parameters to "fully connect" the input and output. While convolutional layers just "slide" the same filters across the input, so it can basically deal with input of an arbitrary spatial size.

In the example network with fully-connected layers at the end, a 224*224 image will output an 1000d vector of class scores. If we apply the network on a larger image, the network will fail because of the inconsistency between the input and parameters of the first fully-connected layer.

One the other hand, if we use a fully convolutional network, when applied to a larger image we'll get 1000 "heatmaps" of class scores.

As shown in the following figure (from the FCN segmentation paper), the upper network gives one score per class, and after the conversion (convolutionalization), we can get the a heatmap per class for a larger image.
enter image description here

About "stride", on the same page, in the section Spatial arrangement:

When the stride is 1 then we move the filters one pixel at a time. When the stride is 2 (or uncommonly 3 or more, though this is rare in practice) then the filters jump 2 pixels at a time as we slide them around. This will produce smaller output volumes spatially.

Improve this answer
$\endgroup$
15
  • 1
    $\begingroup$ @user3269 if the input of the fully-connect layer is 7*7*512, we'll need 7*7*512*4096 parameters to "fully connect" the input and the output. if the spatial size of the input increases, we'll need more parameters. for convolutional layers we just apply the same filter to every part of the input. $\endgroup$
    – dontloo
    Commented Jul 2, 2016 at 7:56
  • 2
    $\begingroup$ @user3269 the network output one value per class for 224*224 images, when we feed in a 384*384 image, it will output a 12*12 heatmap for each class, each pixel of the heatmap is equivalent to the value we get by applying the network on a 224*224 patch in a specific location. because the network reduces the spatial size of he input by a factor of 32 (224*224->7*7), so sliding by one pixel in the top layer is sliding by 32 pixels in the input image. $\endgroup$
    – dontloo
    Commented Jul 2, 2016 at 8:08
  • 1
    $\begingroup$ what if we "slide" on some larger input size with a fc cnn, we could still get a "heatmap". If so, what's the point of a fully conv network? $\endgroup$
    – flankechen
    Commented Dec 5, 2018 at 13:04
  • 1
    $\begingroup$ @flankechen hi, if I understood correctly, sliding an entire cnn to get a heatmap is less efficient since there will be large overlaps among different windows sent into the network, and we will end up applying the same set of filters to these overlapping regions repeatedly. A better choice is to save such redundancy and apply the network to the entire image once, and that can be done by a fully convolutional network. $\endgroup$
    – dontloo
    Commented Dec 5, 2018 at 13:41
  • 1
    $\begingroup$ @flankechen for fully conv networks send the entire image into the network and slide the convolution filters which is more efficient than sliding the whole network, apart from that I think a fully connected network would work as well. $\endgroup$
    – dontloo
    Commented Dec 6, 2018 at 6:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.