I know from theory that LSTM are meant to selectively capture long and short term dependencies in a sequence. I'm trying to implement LSTMs for a time series task and I notice that a lot of tutorials on the web make use of the target sequence lagged by 1 as an input, without including further back observations (at each time step).
What I don't get is whether in this way the distinctive properties of LSTMs as described above are exploited properly. Can LSTM capture long-term dependencies if only fed with the last observation in time? Is this automatically done through the internal state of the LSTM or do I need to feed the network with a window of past time lags into which I want it to capture long/short-term dependencies?