Suppose our network architecture has a hidden layer in which the hidden units are interconnected, then is there some sort of variation on backpropagation that is used? What about in general recurrent neural networks? Backpropagation seems to really take advantage of the "feed-fowardness" of a feed-forward net.

Is this what deep learning is about?


1 Answer 1


The short answer: back-propagation through time. Basically, you can unroll a recurrent neural net and turn it into a feedforward model. You can do this with all sorts of architectures. The architecture that you are describing has been done many times.

For more information on backprop through time: https://www.google.com/url?sa=t&source=web&rct=j&url=http://deeplearning.cs.cmu.edu/pdfs/Werbos.backprop.pdf&ved=0CCsQFjACahUKEwjYuP-44YvGAhWGfpIKHQlyAIM&usg=AFQjCNH6wqddTQOqpIg4_r_kNIz2tV346A&sig2=1DXkilm72s1YphHZAKtv7g

For Yann LeCun's course on deep learning (look for recurrent neural nets) http://cilvr.cs.nyu.edu/doku.php?id=deeplearning2015:schedule

For video lectures: http://techtalks.tv/deep-learning-nyu-spring-2015/


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.