I am trying to classify a particular image.
My labels are PatternA and NotPatternA.
NotPatternA is any image that I have not classified as PatternA. There are very few PatternAs compared to NotPatternA. In my data I have 500 PatternA and 6000 NotPatternA.
I construct my training and testing sets, by choosing 400 from PatternA at random for training and the remaining 100 I put into Testing.
For NotPatternA I randomly draw a sample of 500 images and again split it 400/100 into training and testing.
After 10 epochs I start to get very low loss and high accuracy (near 100%). BUT testing gives me very high loss (2+). However, testing accuracy is also good (near 80%).
Why might my testing loss be so high?