Dynamic graphs versus static graphs in deep learning libraries What is the difference between dynamic graphs and static graphs in deep learning libraries?
Which one is faster?
and when to use each one of them?  
 A: 
What is the difference between dynamic graphs and static graphs in
  deep learning libraries?

For the static graphs, you should first draw the graph completely and then inject data to run(define-and-run), while using dynamic graphs the graph structure is defined on-the-fly via the actual forward computation. This is a far more natural style of programming(define-by-run). 

Which one is faster? and when to use each one of them?

When you can distribute the computations over multiple machines it is better to harness static graphs. Being able to do this with a dynamic computation graph would be far more problematic.
Some other comparisons:
Having a static graph enables a lot of convenient operations: storing a fixed graph data structure, shipping models that are independent of code, performing graph transformations, but it is a little bit more complex than dynamic graphs(for example when implementing something like recursive neural networks). When you're working with new architectures, you want the most flexibility possible, and these frameworks allow for that.
This answer is gathered from the reference at the bottom, and I hope my conclusions are correct and can help. 
Reference: https://news.ycombinator.com/item?id=13428098
