3
$\begingroup$

I am medical student with only a very basic background in CNN's and TensorFlow, so I appreciate any advice.

I implemented a CNN with ~20K black and white images that are 32x32 pixels. I chose to partition my dataset into ~19K images as training data and ~1K images as testing data. Using Tensorflow and TFLearn, I classified my images into one of three classes and plotted accuracy over time with TensorBoard.

The resulting accuracy graph looks like this: CNN accuracy over time

My questions are:

  1. Why does the accuracy suddenly spike down randomly into the mid 90 percents, then shoot back up?
  2. How can I tell if I am "overfitting" my data set?
  3. Is my breakdown of training vs. testing data optimal? (i.e. I chose to use 5% of my images to test and 95% of my images to train. Is there a better combination?)
$\endgroup$
  • $\begingroup$ Is the plot using the training data or the testing data? The units on the x-axis are gradient descent iterations? $\endgroup$ – Matthew Drury Aug 11 '16 at 4:23
  • 1
    $\begingroup$ The plot is using the testing data. Yes, the units on the x-axi are iterations. $\endgroup$ – Andrew J. Aug 11 '16 at 4:28
2
$\begingroup$

Why does the accuracy suddenly spike down randomly into the mid 90 percents, then shoot back up?

It depends on your parameter update strategy, your parameter update strategy's parameters (e.g., learning rate), your network, and your data set.

E.g. see the spike of Rmsprop even though the network had almost converged:

enter image description here

(Images credit: Alec Radford.)

As long as you have the same spike on your validation set, it shouldn't be an issue. If you wish to further investigate, you may want to look at the gradients, and whether the spikes happen on some particular samples.

How can I tell if I am "overfitting" my data set?

See https://datascience.stackexchange.com/a/627/843.

Since it looks like the accuracy on the test set is pretty much 1, I wouldn't worry about overfitting.

Is my breakdown of training vs. testing data optimal? (i.e. I chose to use 5% of my images to test and 95% of my images to train. Is there a better combination?)

The test set should reflect as best as possible the actual samples on which your system will be used in production.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.