I'm using Caffe to train a CNN on a fairly large image set at work. It's about 200K images consisting of profile photos people upload to our app. The labels are either "approve" or "reject" based on guidelines by our community (i.e. no sexual organs, lude finger gestures, etc.). My strategy has been to take a pre-trained model from the Caffe Model Zoo (bvlc_reference_caffenet) and start from there (similar to what is done in the flickr fine tuning example in the example director). No matter what I have tried (wide range of learning rates from 0.1 to 1e-9, freezing several convolution layers, training from scratch, etc), nothing seems to work. The model basically gets to 50% so it's just like flipping a coin (the two classes are balanced 50/50 in the training and test sets). So it's pretty clear the model never actually learns anything.

Furthermore, when I train from scratch, I'm not able to reproduce even the very low-level convolution layers where you normally get edge filters, etc. It's odd to me because my data set isn't too far off from the ImageNet data set which seems to be mostly natural things.

I'm really excited about deep learning, but it's definitely not as simple as I had estimated from the enthusiastic blog posts on it! Seems like more art than science to me right now.

  • $\begingroup$ Maybe this is a silly question, but did you make sure that you centered your inputs to have mean zero? See stats.stackexchange.com/questions/7757/… $\endgroup$ – Flounderer Feb 19 '16 at 1:17
  • $\begingroup$ Definitely not a silly question. In the training file I specified to use the mean from the imagenet photos. I figured that should be pretty close since it's a fairly similar type of data set. Maybe that's not good enough? $\endgroup$ – thecity2 Feb 19 '16 at 1:26
  • 1
    $\begingroup$ Hmm...now I'm wondering about the image labels. I have two classes and I labeled them as "1" and "2". But I just read that they should be 0-indexed. Could this be the root of the issue? $\endgroup$ – thecity2 Feb 19 '16 at 1:34
  • $\begingroup$ I've seen neural networks fail because the input vectors only had positive values in them. That's why I thought it could be the problem. $\endgroup$ – Flounderer Feb 19 '16 at 1:37
  • 1
    $\begingroup$ @IndieAI Yes! That was the issue. $\endgroup$ – thecity2 Feb 22 '16 at 23:02

Turns out as I mentioned in a comment above Caffe expects 0-indexed labels, and I was using 1-indexed (i.e. 1, 2 instead of 0, 1). Hope this helps someone, who might not see it in the documentation and spend a week like I did in frustration.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.