I had wondered the same thing, and it looks like this has been thoroughly researched in this nice blog post:

http://r2rt.com/styles-of-truncated-backpropagation.html ([mirror](https://web.archive.org/web/20170224054758/http://r2rt.com/styles-of-truncated-backpropagation.html))

The author found that the disjoint approach, which is what TensorFlow uses, works reasonably well. Using a sliding window every time step is difficult/expensive to calculate (especially with a framework like TensorFlow), and doesn't yield much benefit. The post cites a paper from the 1990s that found jumping forward h steps, then running BPTT back 2h steps was similar in outcome to just doing the disjoint approach of jumping h steps and running BPTT back also h steps. 

If you're using LSTM's instead of simple RNN cells, then longer contexts do matter and it would probably be better not to truncate at all, or at least use a really high number of unroll steps. The author's results were for a simple RNN model.