# What is an example use of Auto differentiation such as implemented in Tensorflow and why is it important?

I have a decent grasp of neural networks, back propagation and chain rule however I am struggling to understand auto differentiation.

The below refer to auto differentiation outside the context of back propagation:

• How does auto differentiation compute the gradient from a matrix?
• What are the requirements to compute a gradient? Does a function need to be specified?
• What are some use cases for this (other then back propagation)?
• Why is it important and what are the alternatives?

Am I missing something?

• Automatic differentiation, also known as algorithmic differentiation, is an automated way of numerically calculating derivatives of a function(s) specified by a computer program, but the functions can be indirectly defined by the computer program. It is useful for computing gradients, Jacobians, and Hessians for use in numerical optimization, among other things. Backpropagation is an implementation of the reverse mode of automatic differentiation for computing the gradient of a neural network optimization problem See en.wikipedia.org/wiki/Automatic_differentiation . Aug 21, 2016 at 15:03
• Alternatives are symbolic differentiation and finite differences, both of which are usually slower, and finite differences might be less accurate. Hand-coded derivatives can be used if the human can figure out the derivative, but are prone to human mistakes in many cases. Aug 21, 2016 at 15:05
• Where can I find simple examples to see how it is implemented? Is it the tf.gradient method I should be looking at?
– Greg
Aug 21, 2016 at 15:08
• See list of automatic differentiation software and tools at Wikipedia link, and also links which include example usage. Another alternative to automatic differentiation is complex step differentiation (derivative) aero-comlab.stanford.edu/Papers/martins.aiaa.01-0921.pdf , which winds up being almost the same thing as the forward mode of automatic differentiation. Aug 21, 2016 at 15:13