Suppose our network architecture has a hidden layer in which the hidden units are interconnected, then is there some sort of variation on backpropagation that is used? What about in general recurrent neural networks? Backpropagation seems to really take advantage of the "feed-fowardness" of a feed-forward net.

Is this what deep learning is about?


The short answer: back-propagation through time. Basically, you can unroll a recurrent neural net and turn it into a feedforward model. You can do this with all sorts of architectures. The architecture that you are describing has been done many times.

For more information on backprop through time: https://www.google.com/url?sa=t&source=web&rct=j&url=http://deeplearning.cs.cmu.edu/pdfs/Werbos.backprop.pdf&ved=0CCsQFjACahUKEwjYuP-44YvGAhWGfpIKHQlyAIM&usg=AFQjCNH6wqddTQOqpIg4_r_kNIz2tV346A&sig2=1DXkilm72s1YphHZAKtv7g

For Yann LeCun's course on deep learning (look for recurrent neural nets) http://cilvr.cs.nyu.edu/doku.php?id=deeplearning2015:schedule

For video lectures: http://techtalks.tv/deep-learning-nyu-spring-2015/

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.