Questions tagged [automatic-differentiation]

Filter by
Sorted by
Tagged with
0
votes
0answers
14 views

Clarification on notation used to present back propagation algorithm in 'The Deep Learning Book'

In the deep learning book (free version is available online) the backpropation algorithm is explained in section 6.5. I have a question on equation (6.53): $$\frac{\partial u^{(n)}}{\partial u^{(j)}}...
0
votes
0answers
33 views

Gradient of parameterized negative binomial generator?

I have a function negativeBinomial(μ,σ) that generates a random value X that follows the negative binomial distribution of mean $...
1
vote
0answers
83 views

Auto Differentiation in Deep Learning Libraries

It is said that auto-diff is very efficient in generating the derivatives for backpropagation algorithms. The why is it that some of the most widely used deep learning libraries like Theano and ...
7
votes
1answer
5k views

What is an example use of Auto differentiation such as implemented in Tensorflow and why is it important?

I have a decent grasp of neural networks, back propagation and chain rule however I am struggling to understand auto differentiation. The below refer to auto differentiation outside the context of ...
23
votes
1answer
9k views

Step-by-step example of reverse-mode automatic differentiation

Not sure if this question belongs here, but it's closely related to gradient methods in optimization, which seems to be on-topic here. Anyway, feel free to migrate if you think some other community ...