I'm using 3 layer CNN with 8, 16, and 32 filters, each of size 5 X 5. I'm getting an training accuracy of 99.97%. Testing accuracy of 41.11%.
Total classes: 605
Train Set: Each class has 7 samples
Test Set: Each class has 1 sample.
Is the ratio of train and test set causing the huge difference in the train and test accuracy?