1
$\begingroup$

Below given is my cnn-lstm architecture.

model = Sequential()
model.add(TimeDistributed(Conv2D(64, (2, 2), padding='same'), 
                          input_shape=(10,128, 128 ,1))) 
model.add(BatchNormalization())
model.add(Activation("relu"))

model.add(TimeDistributed(MaxPooling2D(pool_size=(2, 2))))
model.add(TimeDistributed(Conv2D(32, (2, 2), padding='same')))
model.add(BatchNormalization())
model.add(Activation("relu"))

model.add(TimeDistributed(MaxPooling2D(pool_size=(2, 2))))
model.add(TimeDistributed(Conv2D(16, (2, 2), padding='same')))
model.add(BatchNormalization())
model.add(Activation("relu"))
model.add(TimeDistributed(MaxPooling2D(pool_size=(2, 2))))

model.add(TimeDistributed(Flatten()))
model.add(LSTM(units=64, return_sequences=True))


model.add(TimeDistributed(Reshape((8, 8, 1))))
model.add(TimeDistributed(UpSampling2D((2,2))))
model.add(TimeDistributed(Conv2D(16, (2, 2), padding='same')))
model.add(BatchNormalization())
model.add(Activation("relu"))
model.add(TimeDistributed(UpSampling2D((2,2))))
model.add(TimeDistributed(Conv2D(32, (2, 2), padding='same')))
model.add(BatchNormalization())
model.add(Activation("relu"))
model.add(TimeDistributed(UpSampling2D((2,2))))
model.add(TimeDistributed(Conv2D(64, (2, 2), padding='same')))
model.add(BatchNormalization())
model.add(Activation("relu"))
model.add(TimeDistributed(UpSampling2D((2,2))))
model.add(TimeDistributed(Conv2D(1, (2, 2), padding='same')))

model.compile(optimizer='RMSProp', loss='mse', metrics=['mean_absolute_error', 'mean_absolute_percentage_error','mean_squared_error','accuracy'])

data = np.load(r"/content/boxing_d1.npy")
print (data.shape)
(x_train,x_test) = train_test_split(data)

x_train = x_train.astype('float32') / 255.
x_test = x_test.astype('float32') / 255.

print (x_train.shape)
print (x_test.shape)
history = model.fit(x_train, x_train,
                epochs=100,
                batch_size=1,
                shuffle=False,
                validation_data=(x_test, x_test))

encoded_imgs = model.predict(x_test)
decoded_imgs = model.predict(encoded_imgs)

I am trying to extract the compressed representation of videos using cnn-lstm inorder to use it for classification purposes using kmeans. Here is the output after a single set of training. Why are the metrics fluctuating as shown in the output. Also i trained a similar model but further training increases the validation and training loss. Ill post a link if someone would like to take a look at it.

model accuracy after first set

model loss after first set

model accuracy after second set

model loss after second set

What is actually wrong here? is it the number of neurons or hidden layers or has it got something to do with the data im feeding?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.