I am trying to train a linear regression model to predict next value of a signal. The network is very simple: it consists of two LSTM cells and a dense layer composed of a single neuron. The optimizer is Adam. The input is populated with the two signal values preceding the target value that is set as output. The records are randomly sorted to each train and input values are normalized with z-score. The input data are about 2 million records while the test data is 100,000. With this configuration the best result I got is a training loss at 0.04 and an evaluation loss at 0.07 (squared error) but I would need to get off a lot more.
I have done countless tests to find the best configuration but the following things seem to make the result worse:
- increase the size of the network
- add dropout
- increase the number of features
- use min-max normalization
- change learning rate / batch size
- change optimizer
I'm probably wrong, but since changing the configuration things do not improve I thought the problem was in the data, so I did some tests transforming the input values (log, exp, sqrt) but without major results. I have also tried to print a plot of data distribution, but I do not know how to interpret them and what transformation to adopt to improve the result.
- blue dots = X: feature #0, Y: target
- purple dots = X: feature #1, Y: target
- cyan dots = X: feature #1, Y: predicted value