6
$\begingroup$

In the paper on ResNet, authors say, that their 152-layer network has lesser complexity than VGG network with 16 or 19 layers:

We construct 101- layer and 152-layer ResNets by using more 3-layer blocks (Table 1). Remarkably, although the depth is significantly increased, the 152-layer ResNet (11.3 billion FLOPs) still has lower complexity than VGG-16/19 nets (15.3/19.6 billion FLOPs)

page 7 top.

How can it be?

$\endgroup$
4
$\begingroup$
input_shape = (3,300,300) # Format:(channels, rows,cols)
conv_filter = (64,3,3,3)  # Format: (num_filters, channels, rows, cols)
stride = 1
padding = 1
activation = 'relu'

n = conv_filter[1] * conv_filter[2] * conv_filter[3]  # vector_length
flops_per_instance = n + 1    # general defination for number of flops (n: multiplications and n-1: additions)

num_instances_per_filter = (( input_shape[1] - conv_filter[2] + 2*padding) / stride ) + 1  # for rows
num_instances_per_filter *= (( input_shape[2] - conv_filter[3] + 2*padding) / stride ) + 1 # multiplying with cols

flops_per_filter = num_instances_per_filter * flops_per_instance
total_flops_per_layer = flops_per_filter * conv_filter[0]    # multiply with number of filters

if activation == 'relu':
    # Here one can add number of flops required
    # Relu takes 1 comparison and 1 multiplication
    # Assuming for Relu: number of flops equal to length of input vector
    total_flops_per_layer += conv_filter[0]*num_instances_per_filter


print(total_flops_per_layer)

This might help you.

$\endgroup$
3
  • 1
    $\begingroup$ An anonymous attempted editor argues that the last line should be: total_flops_per_layer += conv_filter[0]*num_instances_per_filter, because, "In general, flops of relu equals num_filter * output_feature_map_size." $\endgroup$ – gung - Reinstate Monica Aug 5 '18 at 11:31
  • $\begingroup$ @gung Yes, the RELU should be operated on the output matrix. $\endgroup$ – CMCDragonkai Dec 11 '18 at 0:30
  • $\begingroup$ for cols it should be input_shape[2] and conv_filter[3], right? $\endgroup$ – gizzmole Jun 13 '19 at 12:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.