I understood backprop: get gradient wrt a parameter (i.e. the partial derivative) using the chain rule.
In the post http://www.offconvex.org/2016/12/20/backprop/ the authors say that the inefficient way to compute gradient would be quadratic in the terms of nodes of the graph?
I didn't understand what they are trying to say..they are trying to get gradients in feedforward manner? How could that be done even if inefficiently.
So, my question is what is this way of computing gradients inefficiently in a neural net?
Are there other similar techniques too? I always heard backprop was the only way.