I have total of 6300 samples, 5800 of which are training data, and 500 of which are testing data. We compare the performance of LSTM and multilayer perceptron (MLP) with one hidden layer in terms of training process, prediction accuracy and learning ability.
we can observe that RMS error of LSTM is 3.47998, which is less than the MLP of 5.02391. It means LSTM is better than MLP in prediction. Learning ability analysis can be demonstrated by the curve comparison of forecast and observation TEC. We presented the prediction of six days from 2001/3/1 to 2001/3/6 in below Figure. It can be observed that the TEC curve forecasted by LSTM is closely fit for the observed TEC curve which is the ground truth, while there is considerable gap between the curve forecasted by MLP and the ground truth. It indicates MLP is inferior to LSTM in TEC forecast. Nevertheless, the tendency of forecast is right.
My question is: is it possible to find/define appropriate distribution function for both LSTM and MLP based on the above curves in figure?
Update: Consider the above case is Case1.
And there is another two variations of TEC curve below:
Case2: From 2015/6/18-2015/6/23, variation of TEC is very complicated. Beside the periodic variation, there are strong disturbance during one period. On this occasion, LSTM can capture the cyclic changes, and therefore give a better forecast, however, MLP is totally wrong with an inverse direction. Comparing to MLP, LSTM have the advantage of predicting long sequence data due to the memory cell. LSTM can learn the long dependencies of sequential data, not only the very past moment, but also a segment of history is taken into account. While MLP does not utilize history information, so it may fail in case of turbulence of TEC.
Case3:
In case TEC varies suddenly, such as the case illustrated in Fig. 6, where the peaks of observed TEC in the five past days are large, while it dramatically becomes low in the next day. In this situation, RMS error becomes significant.
Above, we discussed three cases of TEC changes, where LSTM all behave better than MLP. The achievement owes to the special design of LSTM, so that it can learn long dependences of sequential data. Thus, the interaction and relationship of the elements of sequential data can be learnt, and the better representation of input data can be obtained.
My question is: what can we say by seeing curves of above three cases for both LSTM and MLP? is it possible to define appropriate distribution function for both LSTM and MLP based on the above curves in figures of three cases? Is there any mathematical representation through distribution possible for three cases?
