1
$\begingroup$

First, let's assume we are in the context of a image classification using a CNN.

I understand that different kernels are applied to the image (or in succession, depending on the depth of the network).

  1. Could you please confirm whether these kernels are updated through back-propagation and training?

  2. If so, would it not be possible to set some of these kernels with predetermined patterns so as to accelerate the training?

thank you

$\endgroup$
1
$\begingroup$

Yes ,these kernels are generally updated via backpropagation when training (although ,you always have a choice of not updating some/all kernels ) .

Also , it is possible to set some of (or all of) these kernels with predetermined 'patterns' to accelerate the learning. This is equivalent to saying that you are using a weight initialization method for your CNN . There could be a variety of ways to do this (this list is no way exhaustive but just to give you an idea).

1) Transfer learning :- For an image classification problem you could use pre-trained kernels from a 'VGG16 architecture', this blog in keras teaches you how to do this .

2) Unsupervised feature learning :- You could also try also learn autoencoders from image patches,use techniques like RICA . Here , you could stack various trained autoencoders (your learned patterns !) and choose not to fine tune (training the complete stacked structure on the training data) and in this case avoid updation through backpropagation. I do not see any reason not to fine tune though.. In case you do not understand this just read this blog (complete it !).

3) Intelligent weight initialisation :- use something like Xaviers initialization or some other heuristic.

This is an open research area in machine learning (you should have a look at this and this ). Intuitively we know that humans can learn from single example's so there must be some kind of one shot learning going on. Hope you find this useful.

$\endgroup$
  • $\begingroup$ Most excellent comments, Vaibhav. I have used CNN at a general level but with the info you have provided I feel it is possible to explore further all kind of very neat ideas. It is exactly what I needed. Thanks again! $\endgroup$ – Gilles Daquin Mar 17 '17 at 2:32
  • $\begingroup$ glad to help !! also there are pretty good lectures by stanford on neural networks for image recognition , i think you will find them interesting !! you can find them at youtube.com/… . if you liked my answer please accept/upvote helps with the reputation :) $\endgroup$ – Vaibhav Arora Mar 17 '17 at 5:09
  • $\begingroup$ I have voted up your remark and it has been counted but my 'reputation' as a new user being still below 15, the vote is not publicly displayed yet. It is counted though. Sorry for being new on the stack and that absolutely does not diminish your help to another person. THANK YOU! $\endgroup$ – Gilles Daquin Apr 4 '17 at 4:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.