I am reading Stanford's tutorial on the subject, and I have reached this part, "Training a Neural Network". So far so good. I understand pretty much everything.
I would like to change the ReLU he is using there, with a Leaky ReLU. My question, is, do I have to change the way he is doing the back-propagation? How do these derivatives going to change if I use a Leaky ReLU?
Any paper that's states exactly how back prop is done when we have a Leaky ReLU?