1
$\begingroup$

So I understand, that the main advantage from CNNs are that there are less leanrable parameters and that this should speed up learning. Now I build two models to train on the fashion_mnist dataset. These are the Models I created:

def build_model_seq(input_shape,output_shape):
    model = tf.keras.models.Sequential()
    model.add(tf.keras.layers.Flatten(input_shape=input_shape))
    model.add(tf.keras.layers.Dense(units=64,activation='relu'))
    model.add(tf.keras.layers.Dense(units=64,activation='relu'))
    model.add(tf.keras.layers.Dense(units=output_shape,activation='softmax'))

    opt = tf.keras.optimizers.SGD()
    
    model.compile(  optimizer=opt,
                    loss='sparse_categorical_crossentropy',
                    metrics=['accuracy'])
    

    return model

def build_model_cnn(input_shape,output_shape):
    model = tf.keras.Sequential()
    model.add(tf.keras.layers.Conv2D(filters=16,kernel_size=(3,3),
                                        activation='relu',input_shape=input_shape))
    model.add(tf.keras.layers.MaxPool2D())
    model.add(tf.keras.layers.Conv2D(filters=8,kernel_size=(3,3),
                                        activation='relu'))
    model.add(tf.keras.layers.MaxPool2D())
    model.add(tf.keras.layers.Flatten())
    model.add(tf.keras.layers.Dense(units=output_shape,activation='softmax'))

    opt = tf.keras.optimizers.SGD()
    
    model.compile(  optimizer=opt,
                    loss='sparse_categorical_crossentropy',
                    metrics=['accuracy'])

    return model

Now If I train these models the fitting time on a single epoch is 8 times larger. This can be seen here: enter image description here

Why is that? Can someone explain this to me please and after that the overall benefit of CNNs other than they just work better?

$\endgroup$
1
$\begingroup$

Convolutional neural networks are very different from the standard feedforward neural networks. CNN use kernels that seek for features on a different part of an image (or sequence, or another type of data, since there are also CNN's for non-image data). The consequence is that they need to make much more computations than the dense layers, so obviously, they would be slower. We use them because applying a kernel to different parts of an image lets us discover the same patterns on different parts of an image. If your training data contains only the images with cars in the upper left part of an image, a feedforward network would know how to detect cars only if they appeared in exactly the same region of an image, while CNN would generalize to discovering them no matter where they are located.

$\endgroup$
2
  • $\begingroup$ And why are the number of parameters such a big factor when advertising CNNs? $\endgroup$
    – HWilmer
    May 20 at 13:07
  • $\begingroup$ @HWilmer I'm not sure if I understand your question. $\endgroup$
    – Tim
    May 20 at 13:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.