Why use LSTM or in general RNN instead of plain NN? I understand that in LSTM, there are 3 gates to help keep the memory. But why do we use LSTM instead of NN in the first place?
For example, in a LSTM, my features are just 1-D, time steps is set to be 60, which means the longest memory my LSTM can remember is just 60 days ago, you can't use any information 73 days ago to predict today's target.
But in a plain NN, we can reconstruct the 1-D variable into a 60-D variables with the other dimensions being the variable's value 1-day ago, 2-day ago, ..., 60-day ago. And we predict today's target variable.
What is the difference here? We are all just using 60-day information to predict a single variable.
 A: A couple of reasons:
First, you might not know how long your sequence will be ahead of time. RNNs let you use a variable length input.
Second, RNNs are a good way to do parameter sharing between different time steps. Instead of having to learn how to extract useful information from each time step separately you can reuse the same parameter matrix recurrently.
A: First to clarify: There are 3 types of NNets used for this type of problem. Feed forward NNets (i.e plain NN), Recurrent NNets (RNN), and LSTM, which is a more advanced class of RNN. 
The problem with plain NN is that they don't have any memory. Let's take your example of: 

But in a plain NN, we can reconstruct the 1-D variable into a 60-D variables with the other dimensions being the variable's value 1-day ago, 2-day ago, ..., 60-day ago. And we predict today's target variable.

Now say there are recursive relationships between you data points, so that the Data(Today) = f[Data(Yesterday)] = f[f[Data(2 Days ago)]], etc...
A plain NN can learn the relationship between these data points during the training phase, but once the training is done, it has no way of applying such recursive relationships for future data points because the data between the inputs and the outputs flows in only one direction.  
Putting it another way, a plain NN can predict the value for today. But if you wanted to predict the target variable for 2 days, 3 days, or N days ahead (see one-step ahead vs. multi-sept forecasting) - then the NN has no way of predicting those values, since it has no way of feeding the forecast for today back as an input to generate a forecast for tomorrow and the day after.
RNN and LSTM on the other hand, have feedback loops going from the outputs back to the inputs, which allow them to perform multistep forecasting. 
Moreover, LSTM have additional properties that allow them to learn long term dependencies, whereas regular RNN can only learn short dependencies. 
