Cannot overfit Mobilenet with one example

Question

I am trying to do a single object detection. Since the problem is much simpler than multibox object localization I decided to try using a simple CNN that predicts the object class and its location. Simple neural net for this task in Keras looks like

inpt = Input(shape=input_shape, name='input_image')
h = Conv2D(32, 3, 3, padding='same', activation='relu')(inpt)
h = MaxPooling2D(pool_size=(2, 2))(h)

h = Conv2D(64, 3, 3, padding='same', activation='relu')(h)
out = MaxPooling2D(pool_size=(2, 2))(h)

out = GlobalAveragePooling2D()(out)

coordinates = Dense(4)(out)
classes = Dense(2, activation='softmax')(classes)

model = Model(inputs=inpt, outputs=concatenate([coordinates, classes]))

Here I reduced the problem to detect an object of one type or background - binary classifier.

Now I want to make sure my pipeline is set up correctly so I want to overfit the model with several examples - train. I am able to do it. Next I want to take a bigger net as the base, for example, Mobilenet. And the problem is that I can't overfit it even on one example. Any thoughts?

Answer 1

It turned out the batch normalization layer was the reason. Removing it allowed me to overfit on one example.