I am not sure I am thinking about my problem the right way, so I am looking for the right approach.

I have a data set that, for the sake of argument, has a mean of 1 and a standard deviation of $\sigma$. It is time series data and I have converted it so that it is stationary and then trained a LSTM model to predict the next value $x$ from the previous $n$ observations where $n$ is relatively small, say 15 observations. When I evaluate the model, I capture the error for each prediction along with MSE and MAE, but the model uses the MAE as a loss function.

My task is to predict $x$ and compare it to an observed value. If the observed value is 'too far' away from the prediction, I want to check it as it may be a bad value. What I am struggling with is how to define 'too far'. Intuitively, I want to define 'too far' as 3 standard deviations of error away from the predicted value of $x$. But this somehow doesn't seem right, or at least, I don't see much discussion of this approach.

My question is, should I compute the standard deviation of the error and define my test using this measure, or should I use the standard deviation of the raw data? If I can use the standard deviation of the error, can/should I calculate it based on the MAE or the raw error values?

Any guidance would be helpful.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.