Why is Hebbian learning a less preferred option for training deep neural networks? I understand that backpropagation is good, but what are the main advantages (and disadvantaged) that it has over Hebbian learning?
I'm mostly wondering about contrastive Hebbian learning, though arguments against Hebbian learning in general are welcomed. 
 A: I have been doing simple toy experiments for a while and I find it is rather difficult to make Hebbian rules work well. Fundamentally, Hebbian learning leans more towards unsupervised learning as a teacher signal at a deep layer cannot be efficiently propagated to lower levels as in backprop with ReLu. This in turn means that for Hebbian systems to be applied to real problems, the system has to find 'good' representations that can easily be 'fetched' by another supervised algorithm as in the following recent example:
http://www.ece.ucsb.edu/wcsl/people/aseem/Aseem_stuff/hebbian_preprint.pdf
As for advantages of Hebbian learning I would list the following:


*

*high online adaptivity to changes in the input distribution

*(probably) higher suitability to super-deep architectures, especially RNNs

*biological plausibility

*easy interpretability of what the algorithm is doing layer-to-layer


I hope others with more knowledge will chime in for some comments as I'd like to know more about this as well.
