# Chinese character recognition from generated images - Validation accuracy does not improve

I am currently working on creating a simple Chinese character recognition network. Given an grayscale image of a character, the goal is to predict the depicted character. I want to run the model on a smartphone, so I use the tensorflow 2.0 keras MobileNet 2. I did not find suitable datasets with a reasonable large resolution, so I downloaded lots of fonts and generate it. Pm average, there are 50 images per class.

Found 5295 images belonging to 100 classes.
Found 1282 images belonging to 100 classes.


There are many Chinese characters, but I started to validate my approach on the 100 most frequent first.

My problem now is that the training error goes down nicely, but the validation error stays at 1%, which is random chance with the number of classes.

83/83 [==============================] - 395s 5s/step - loss: 4.1869 - accuracy: 0.0721 - val_loss: 4.6041 - val_accuracy: 0.0109
83/83 [==============================] - 334s 4s/step - loss: 1.5779 - accuracy: 0.5626 - val_loss: 4.6264 - val_accuracy: 0.0109
83/83 [==============================] - 339s 4s/step - loss: 0.3192 - accuracy: 0.9126 - val_loss: 4.6564 - val_accuracy: 0.0109
83/83 [==============================] - 331s 4s/step - loss: 0.1510 - accuracy: 0.9617 - val_loss: 4.7007 - val_accuracy: 0.0109


What I tried:

• Played with dropout
• Played with the alpha of the MobileNet
• Data augmentation

I observed that the model learns better when I load the pictures as RGB instead of grayscale. For my use case, I just need binary images, therefore using RGB seems strange. The same is true when I generate the images as RGB with a random paper texture background. Does anyone know what is going on there? I know that I likely have not enough data, especially for scaling up the number of classes, but I need to first tackle the stagnating validation accuracy.

My model code:

import tensorflow as tf

import matplotlib.pyplot as plt
from tensorflow.python.keras.callbacks import EarlyStopping

from yangtao.config import PATH_DATA_RESULTS, PATH_DATA_GENERATED_HANZI

PATH_DATA_RESULTS.mkdir(parents=True, exist_ok=True)

DATA_TRAIN_DIR = PATH_DATA_GENERATED_HANZI

IMAGE_SIZE = 96
BATCH_SIZE = 64
IMG_SHAPE = (IMAGE_SIZE, IMAGE_SIZE, 1)

datagen = tf.keras.preprocessing.image.ImageDataGenerator(
preprocessing_function=tf.keras.applications.mobilenet_v2.preprocess_input,
validation_split=0.2,
rotation_range=5,
shear_range=0.01,)

train_generator = datagen.flow_from_directory(
DATA_TRAIN_DIR,
target_size=(IMAGE_SIZE, IMAGE_SIZE),
batch_size=BATCH_SIZE,
color_mode="grayscale",
subset='training')

val_generator = datagen.flow_from_directory(
DATA_TRAIN_DIR,
target_size=(IMAGE_SIZE, IMAGE_SIZE),
batch_size=BATCH_SIZE,
color_mode="grayscale",
subset='validation')

labels = '\n'.join(sorted(train_generator.class_indices.keys()))
number_of_classes = len(train_generator.class_indices)

with open(PATH_DATA_RESULTS / 'labels.txt', 'w') as f:
f.write(labels)

# Create the base model from the pre-trained model MobileNet V2
base_model = tf.keras.applications.MobileNetV2(input_shape=IMG_SHAPE,
include_top=False,
weights=None,
alpha=0.75)

model = tf.keras.Sequential([
base_model,
# tf.keras.layers.Conv2D(32, 3, activation='relu'),
tf.keras.layers.Dropout(0.5),
tf.keras.layers.GlobalAveragePooling2D(),
tf.keras.layers.Dense(number_of_classes, activation='softmax')
])

# model = cnn_indian()
loss='categorical_crossentropy')

epochs = 100

earlystop_callback = EarlyStopping(
monitor='val_accuracy', min_delta=0.0001,
patience=5)

history = model.fit(train_generator,
epochs=epochs,
verbose=1,
validation_data=val_generator,
callbacks=[earlystop_callback])


The generated images look like the following (I have 100 different classes with 50 images each):

• The validation loss increases (see the log snippet) – reindeer Aug 12 at 17:45