# Accuracy of Keras Model is Very Low for Identifying Differently Colored Objects

I am using transfer learning approach to train my keras model to identify objects which have same structure but the colors are different i.e objects are to be identified by their respective color.

Model used = vgg16

Total object classes = 7

Number of training images = 2100

Number of Validation images = 525

Image size = (224, 224)

Following is my model compilation details:

model.compile(Adam(lr = .0001), loss = 'categorical_crossentropy', metrics=['accuracy'])


Number of epochs = 500

Code Snippet:

model.layers.pop()  #removed the last layer of the pre-trained model
#adding output layer as per my requirement

# Here reading my train, test and validation dataset
train_batches = ImageDataGenerator().flow_from_directory(train_path, target_size = (224,224), classes = ['N52204400D','N52204403M','N52204406G','N52204500DA','N52204503M','N52204506G','N522044064A'],batch_size = 20)
valid_batches = ImageDataGenerator().flow_from_directory(valid_path, target_size = (224,224), classes = ['N52204400D','N52204403M','N52204406G','N52204500DA','N52204503M','N52204506G','N522044064A'],batch_size = 5)
test_batches = ImageDataGenerator().flow_from_directory(test_path, target_size = (224,224), classes = ['N52204400D','N52204403M','N52204406G','N52204500DA','N52204503M','N52204506G','N522044064A'],batch_size = 7)

#Finally Training the new model
model.compile(Adam(lr = .0001), loss = 'categorical_crossentropy', metrics=['accuracy'])
model.fit_generator(train_batches, steps_per_epoch=<>, validation_data=valid_batches, validation_steps=<>, epochs=<>, verbose=2)


Issue faced:

Accuracy of my model is not improving beyond 60%.

Is there anything I am missing? Do I need to tweak my code?If yes, then how? Or if there is any better way to train a model to identify differently colored object, please advice.