My convolutional network seems to work well in learning the features. However, the accuracy of the validation set is increasing very slowly with respect to the learning rate as also illustrated in the figure below:
The loss of both training and validation sets are shown in the figure below:
If I decrease the learning rate, the validation accuracy will stay around 25% and it will not increase. Is there any method to speed up the validation accuracy increment while decreasing the rate of learning?