0
$\begingroup$

I am building an LSTM Autoencoder (unsupervised model) to detect anomalies in a time series dataset. The input is telemetry data from routers and I want to detect anomalies in the throughout of routers. I am evaluating the model using MAE (Mean Absolute Error). Let's say I split my entire dataset of 1000 rows into time windows of 30 days. Data in each row corresponds to 1 day. So the first time window will contain data from the first 30 days (row 1 to row 30). This window will be used to predict the value for the 31st day. Similarly, the time window will change by a day on a rolling basis as I predict the value for the 32nd day, 33rd day,..etc. Once the time windows are created, the model predicts the values and then compares them with the actual values. The model tries to minimize the Mean absolute Error (MAE) between the predicted and actual values. MAE is calculated for each predicted value. I have 3 questions:

  1. How do I set the threshold for MAE? E.g. All predicted values with MAE > 0.05 are anomalies. How do I decide if the threshold should be 0.05 or 0.03 etc.?

  2. Once I have created version 1 of the model (version A), I will retrain it and create version B. Should I keep the same threshold for both model versions?

  3. How do I determine if version B is better than version A?

$\endgroup$
2
  • $\begingroup$ (1) MAE is a number, by definition. Please explain how you obtain a histogram. And please, no, I am not going to watch a video to try to figure what you might mean. $\endgroup$
    – whuber
    May 9 at 13:37
  • $\begingroup$ @whuber - I have edited the question to add clarity, explain my process, and specify in detail what I'm looking for. Kindly open the question for answers. $\endgroup$ May 10 at 2:08

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.