# How to understand / calculate FLOPs of the neural network model?

In the paper on ResNet, authors say, that their 152-layer network has lesser complexity than VGG network with 16 or 19 layers:

We construct 101- layer and 152-layer ResNets by using more 3-layer blocks (Table 1). Remarkably, although the depth is significantly increased, the 152-layer ResNet (11.3 billion FLOPs) still has lower complexity than VGG-16/19 nets (15.3/19.6 billion FLOPs)

page 7 top.

How can it be?

input_shape = (3,300,300) # Format:(channels, rows,cols)
conv_filter = (64,3,3,3)  # Format: (num_filters, channels, rows, cols)
stride = 1
activation = 'relu'

n = conv_filter * conv_filter * conv_filter  # vector_length
flops_per_instance = n + 1    # general defination for number of flops (n: multiplications and n-1: additions)

num_instances_per_filter = (( input_shape - conv_filter + 2*padding) / stride ) + 1  # for rows
num_instances_per_filter *= (( input_shape - conv_filter + 2*padding) / stride ) + 1 # multiplying with cols

flops_per_filter = num_instances_per_filter * flops_per_instance
total_flops_per_layer = flops_per_filter * conv_filter    # multiply with number of filters

if activation == 'relu':
# Here one can add number of flops required
# Relu takes 1 comparison and 1 multiplication
# Assuming for Relu: number of flops equal to length of input vector
total_flops_per_layer += conv_filter*num_instances_per_filter

print(total_flops_per_layer)


• An anonymous attempted editor argues that the last line should be: total_flops_per_layer += conv_filter*num_instances_per_filter, because, "In general, flops of relu equals num_filter * output_feature_map_size." – gung - Reinstate Monica Aug 5 '18 at 11:31
• for cols it should be input_shape and conv_filter, right? – gizzmole Jun 13 '19 at 12:44