I am currently working on creating a simple Chinese character recognition network. Given an grayscale image of a character, the goal is to predict the depicted character. I want to run the model on a smartphone, so I use the tensorflow 2.0 keras MobileNet 2. I did not find suitable datasets with a reasonable large resolution, so I downloaded lots of fonts and generate it. Pm average, there are 50 images per class.
Found 5295 images belonging to 100 classes. Found 1282 images belonging to 100 classes.
There are many Chinese characters, but I started to validate my approach on the 100 most frequent first.
My problem now is that the training error goes down nicely, but the validation error stays at 1%, which is random chance with the number of classes.
83/83 [==============================] - 395s 5s/step - loss: 4.1869 - accuracy: 0.0721 - val_loss: 4.6041 - val_accuracy: 0.0109 83/83 [==============================] - 334s 4s/step - loss: 1.5779 - accuracy: 0.5626 - val_loss: 4.6264 - val_accuracy: 0.0109 83/83 [==============================] - 339s 4s/step - loss: 0.3192 - accuracy: 0.9126 - val_loss: 4.6564 - val_accuracy: 0.0109 83/83 [==============================] - 331s 4s/step - loss: 0.1510 - accuracy: 0.9617 - val_loss: 4.7007 - val_accuracy: 0.0109
What I tried:
- Played with dropout
- Played with the alpha of the MobileNet
- Data augmentation
I observed that the model learns better when I load the pictures as RGB instead of grayscale. For my use case, I just need binary images, therefore using RGB seems strange. The same is true when I generate the images as RGB with a random paper texture background. Does anyone know what is going on there? I know that I likely have not enough data, especially for scaling up the number of classes, but I need to first tackle the stagnating validation accuracy.
My model code:
import tensorflow as tf import matplotlib.pyplot as plt from tensorflow.python.keras.callbacks import EarlyStopping from yangtao.config import PATH_DATA_RESULTS, PATH_DATA_GENERATED_HANZI PATH_DATA_RESULTS.mkdir(parents=True, exist_ok=True) DATA_TRAIN_DIR = PATH_DATA_GENERATED_HANZI IMAGE_SIZE = 96 BATCH_SIZE = 64 IMG_SHAPE = (IMAGE_SIZE, IMAGE_SIZE, 1) datagen = tf.keras.preprocessing.image.ImageDataGenerator( preprocessing_function=tf.keras.applications.mobilenet_v2.preprocess_input, validation_split=0.2, rotation_range=5, shear_range=0.01,) train_generator = datagen.flow_from_directory( DATA_TRAIN_DIR, target_size=(IMAGE_SIZE, IMAGE_SIZE), batch_size=BATCH_SIZE, color_mode="grayscale", subset='training') val_generator = datagen.flow_from_directory( DATA_TRAIN_DIR, target_size=(IMAGE_SIZE, IMAGE_SIZE), batch_size=BATCH_SIZE, color_mode="grayscale", subset='validation') labels = '\n'.join(sorted(train_generator.class_indices.keys())) number_of_classes = len(train_generator.class_indices) with open(PATH_DATA_RESULTS / 'labels.txt', 'w') as f: f.write(labels) # Create the base model from the pre-trained model MobileNet V2 base_model = tf.keras.applications.MobileNetV2(input_shape=IMG_SHAPE, include_top=False, weights=None, alpha=0.75) model = tf.keras.Sequential([ base_model, # tf.keras.layers.Conv2D(32, 3, activation='relu'), tf.keras.layers.Dropout(0.5), tf.keras.layers.GlobalAveragePooling2D(), tf.keras.layers.Dense(number_of_classes, activation='softmax') ]) # model = cnn_indian() model.compile(optimizer=tf.keras.optimizers.Adam(), loss='categorical_crossentropy') epochs = 100 earlystop_callback = EarlyStopping( monitor='val_accuracy', min_delta=0.0001, patience=5) history = model.fit(train_generator, epochs=epochs, verbose=1, validation_data=val_generator, callbacks=[earlystop_callback])
The generated images look like the following (I have 100 different classes with 50 images each):