0
$\begingroup$

i'm still new in machine learning. currently i'm creating an anomaly detection for flight data. it is a multivariate time series data that include timestamp, latitude, longitude, velocity and altitude of the aircraft. i'm splitting the data into train and test with 80% ratio. i used the keras LSTM autoencoder to do a anomaly detection. so here's a snippet of my code

def create_sequence(data, time_step = None):
    Xs = []
    
    for i in range (len(data) - time_step):
        Xs.append(data[i:(i + time_step)])

    return np.array(Xs)

# pre-process to split the data

dfXscaled, scalerX = scaledf(df, normaltype=normalization)
num_train = int(df.shape[0]*ratio)

values_dataset = dfXscaled.values

train = values_dataset[:num_train, :]
test = values_dataset[num_train:, :]

# sequence input data [sample, time step, features]
train_input = create_sequence(train, time_step = time_step) 
test_input = create_sequence(test, time_step = time_step) 

train_time = index_time.index[:num_train]
test_time = index_time.index[num_train:]

# model 
model_arch = []

last_layer = num_layers - 1
for x in range(num_layers):
    if x == last_layer:
        model_arch.append(tf.keras.layers.LSTM(num_nodes, activation='relu', return_sequences=True, dropout = dropout))
    else:
        model_arch.append(tf.keras.layers.LSTM(num_nodes, activation='relu', input_shape=(time_step, 4), dropout = dropout))  
        model_arch.append(tf.keras.layers.RepeatVector(time_step))
        
model_arch.append(tf.keras.layers.TimeDistributed(tf.keras.layers.Dense(4)))
model = tf.keras.models.Sequential(model_arch)
opt= tf.keras.optimizers.SGD(learning_rate=learning_rate)
model.compile(loss=tf.keras.losses.Huber(),
              optimizer=opt,
              metrics=[tf.keras.metrics.MeanAbsolutePercentageError(name='mape'), tf.keras.metrics.RootMeanSquaredError(name='rmse'), "mae", 'accuracy'])
history = model.fit(train_input, train_input, epochs=epochs, batch_size = num_batch, validation_data=(test_input, test_input), verbose=2, shuffle=False)

when i do a model evaluation, it come up with 100% accuracy model evaluation

is it good to have a 100% on validation ? or my model is overfitting ? any answer would be appreciated. thank you

$\endgroup$
2
  • $\begingroup$ Your code says scores = model.evaluate(test_input, test_input) - that makes me suspect that you are not actually testing the model, just passing the ground thruth in. $\endgroup$
    – Jon Nordby
    Commented Jun 3, 2022 at 20:30
  • $\begingroup$ Also, why are you using accuracy for what seems to be a regression problem? $\endgroup$
    – Jon Nordby
    Commented Jun 3, 2022 at 20:31

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.