I have a decent grasp of neural networks, back propagation and chain rule however I am struggling to understand auto differentiation.
The below refer to auto differentiation outside the context of back propagation:
- How does auto differentiation compute the gradient from a matrix?
- What are the requirements to compute a gradient? Does a function need to be specified?
- What are some use cases for this (other then back propagation)?
- Why is it important and what are the alternatives?
Am I missing something?
tf.gradient
method I should be looking at? $\endgroup$