I am not sure I am thinking about my problem the right way, so I am looking for the right approach.
I have a data set that, for the sake of argument, has a mean of 1 and a standard deviation of $\sigma$. It is time series data and I have converted it so that it is stationary and then trained a LSTM model to predict the next value $x$ from the previous $n$ observations where $n$ is relatively small, say 15 observations. When I evaluate the model, I capture the
error for each prediction along with
MAE, but the model uses the
MAE as a loss function.
My task is to predict $x$ and compare it to an observed value. If the observed value is 'too far' away from the prediction, I want to check it as it may be a bad value. What I am struggling with is how to define 'too far'. Intuitively, I want to define 'too far' as 3 standard deviations of error away from the predicted value of $x$. But this somehow doesn't seem right, or at least, I don't see much discussion of this approach.
My question is, should I compute the standard deviation of the error and define my test using this measure, or should I use the standard deviation of the raw data? If I can use the standard deviation of the error, can/should I calculate it based on the
MAE or the raw
Any guidance would be helpful.