I am newbie in neural networks and I am trying to build a LSTM model to predict future values. My problem is that the plot of predictions result returns a line in comparation with the testting data.
I alredy search for a solution before asking this question and that migth help me was this one. The answer given by "OverLordGodDragon" also have a link to another question with an answer that he give explaining how to feed a LTSM model. That was quite interesting and usefull to have it in mind. But even so I still lost.
The plot that my model give is not a straigth line but even that the behaviur is wrong, my result:
One thing to have in mind is that I am working with a small part of the data because of the huge amount it is, about 5 mill, and to prevent the kernel die I work with about 5 thousend of that. The data I loaded from a CSV and I take it by the sample() function of pandas using "frac" parameter. Now I will show my code:
Scaling and normalazing between -1 an 1:
# Feature Scaling Normalization
scaler = preprocessing.MinMaxScaler()
# min-max normalization and scale the features in the 0-1 range.
ifv_values = df['IFVα'].values.reshape(-1, 1)
# The scaler expects the data to be shaped as (x, y)
scaled_ifv = scaler.fit_transform(ifv_values)
# removing NaNs (if any)
scaled_ifv = scaled_ifv[~np.isnan(scaled_ifv)]
# reshaping data after removing NaNs
scaled_ifv = scaled_ifv.reshape(-1, 1)
Spliting the data in trainable data and testing:
SEQ_LEN = 100
def to_sequences(data, seq_len):
d = []
for index in range(len(data) - seq_len):
d.append(data[index: index + seq_len])
return np.array(d)
def preprocess(data_raw, seq_len, train_split):
data = to_sequences(data_raw, seq_len)
num_train = int(train_split * data.shape[0])
X_train = data[:num_train, :-1, :]
y_train = data[:num_train, -1, :]
X_test = data[num_train:, :-1, :]
y_test = data[num_train:, -1, :]
return X_train, y_train, X_test, y_test
# 20% of the data saved for testing.
X_train, y_train, X_test, y_test = preprocess(scaled_ifv, SEQ_LEN, train_split = 0.80)
print(X_train.shape, y_train.shape, X_test.shape, y_test.shape)
>>(4337, 99, 1) (4337, 1) (1085, 99, 1) (1085, 1)
My model structure:
DROPOUT = 0.2
UNITS = SEQ_LEN - 1
model = Sequential()
# Input layer
model.add(LSTM(UNITS, activation='tanh',input_shape=(UNITS,
X_train.shape[-1]),return_sequences=True))
model.add(Dropout(rate=DROPOUT))
# 1st Hidden layer
model.add(LSTM(UNITS, return_sequences = True))
model.add(Dropout(rate=DROPOUT))
# 2nd Hidden layer
model.add(LSTM(UNITS, return_sequences=False))
# output layer
model.add(Dense(units=1))
model.add(Activation('linear'))
And the compiling and training code, I use MSE as loss function and the Adam optimizator:
BATCH_SIZE = 64
lstm_model.compile(loss='mean_squared_error', optimizer='adam')
my_callbacks =[
save_checkpoint_foreach_epoch('history_lstm_model'),
#save_model_folder ('model_lstm'),
]
history_lstm = lstm_model.fit(X_train, y_train,
batch_size= BATCH_SIZE,
epochs=50,
shuffle=False,
callbacks= my_callbacks,
validation_split=0.1,
verbose=2)
I appreciate any help.Thanks for your time.