How to understand / calculate FLOPs of the neural network model?

Question

In the paper on ResNet, authors say, that their 152-layer network has lesser complexity than VGG network with 16 or 19 layers:

We construct 101- layer and 152-layer ResNets by using more 3-layer blocks (Table 1). Remarkably, although the depth is significantly increased, the 152-layer ResNet (11.3 billion FLOPs) still has lower complexity than VGG-16/19 nets (15.3/19.6 billion FLOPs)

page 7 top.

How can it be?

Answer 1

input_shape = (3,300,300) # Format:(channels, rows,cols)
conv_filter = (64,3,3,3)  # Format: (num_filters, channels, rows, cols)
stride = 1
padding = 1
activation = 'relu'

n = conv_filter[1] * conv_filter[2] * conv_filter[3]  # vector_length
flops_per_instance = n + 1    # general defination for number of flops (n: multiplications and n-1: additions)

num_instances_per_filter = (( input_shape[1] - conv_filter[2] + 2*padding) / stride ) + 1  # for rows
num_instances_per_filter *= (( input_shape[2] - conv_filter[3] + 2*padding) / stride ) + 1 # multiplying with cols

flops_per_filter = num_instances_per_filter * flops_per_instance
total_flops_per_layer = flops_per_filter * conv_filter[0]    # multiply with number of filters

if activation == 'relu':
    # Here one can add number of flops required
    # Relu takes 1 comparison and 1 multiplication
    # Assuming for Relu: number of flops equal to length of input vector
    total_flops_per_layer += conv_filter[0]*num_instances_per_filter


print(total_flops_per_layer)

This might help you.