2
$\begingroup$

I am currently working on a univariate time series data and I wanted to know if anomaly forecasting is possible in time series.

I previously worked on anomaly detection which detects the anomaly when happened(basically uses the ground truth for detection). But now I got a relatively challenging task where anomaly forecasting has to be done before it happens. Since it is an anomaly there won't be a trend and the class imbalance is obvious.

If it was not time series data, then some sampling techniques such as oversampling and undersampling would have helped balance the anomalous and non-anomalous data classes (like fraud detection in emails). Since it is time series data, there is a possibility of getting duplicate time stamps when sampled.

I would really appreciate it if someone could give some insights on this.

$\endgroup$
9
  • $\begingroup$ Welcome to Cross Validated! Is there anything leading up to an anomaly to suggest that an anomaly is likely to occur? $\endgroup$
    – Dave
    Commented Sep 17 at 18:23
  • $\begingroup$ Before I tell you what are the basis of anomaly to be an anomaly, I would like to tell the approach we followed earlier during anomaly detection. Assume an example of a CPU's usage which has gone past the dynamic threshold(upper threshold and lower threshold) of the based on previous window's ground truth and forecast values. In this case we are using the difference between the threshold and the ground truth values and if the difference is negative then it is an anomaly. We are predicting this difference values and trying to forecast these differences. $\endgroup$ Commented Sep 18 at 3:49
  • $\begingroup$ But does anything happen leading up to the anomalies to suggest that a forthcoming anomaly is likely? Your question remarks that there is not a trend. If that is the case, then why should you be able to reliably forecast anomalies? (I am thinking of the difference between marginal and conditional anomalies.) $\endgroup$
    – Dave
    Commented Sep 18 at 12:09
  • $\begingroup$ With the data I have there is no observable trend that suggests the happening of an anomaly. Since it is univariate time series data, the case of conditional anomalies can be ignored. Does this mean the quality of the data I have in hand is the real issue? $\endgroup$ Commented Sep 18 at 17:27
  • $\begingroup$ The fact that there is no observable trend is not ideal but is not a dealbreaker, as a good predictive model may be able to pick up on trends that are not evident upon visual examination. However, if there really is nothing in the past that suggests an anomaly is likely forthcoming, you are in trouble: why would you ever want to predict a forthcoming anomaly instead of business as usual when the observed trend is business as usual? Perhaps you would need to look at multiple time series, even if the anomalies of interest are in just the one time series. $\endgroup$
    – Dave
    Commented Sep 18 at 18:02

1 Answer 1

1
$\begingroup$

Since it is an anomaly there won't be a trend

Depending on how literal you are about this statement, this might say it all.

The literal meaning of this is that nothing in your data is predictive of these anomalies. If that is the case, then the problem is, in some sense, impossible to solve. The conditional probability of an anomaly, conditioned on all present information (the entire history of the time series), says that an anomaly is unlikely and no more likely than at any other time with any other time series history. Nothing distinguishes the anomalies from the "business-as-usual" non-anomalies. In this case where the problem seems hopeless, if you want to make good forecasts of anomalies, you will have to bring in additional information, perhaps another time series that, alone or along with the current time series, is predictive of anomalies of interest. In the link, that additional information is when Easter occurs.

A less pessimistic read of the claim is that the trends might not be obvious but, you hope, are there for a sophisticated machine learning model to find. In that case, even if you cannot look at the time series and say, credibly and reliably, "An anomaly is likely to occur in 20-30 seconds," a good model might be able to pick up on trends that elude manual inspection.

In that case, you do have some information to credibly and reliably predict anomalies. While the marginal or overall probability of an anomaly might be very low (anomalies are uncommon, after all), the conditional probability of an anomaly is higher than usual. However, because of how unusual anomalies are, this probability might still be rather low, perhaps only $0.3$ instead of a baseline (overall) probability of $0.00003$.

This seems to be where you would want to use oversampling in order to boost the probability of an anomaly occurring in order to get a value above $0.5$, probably because a software package will then predict this as an anomaly when you run some kind of predict method. Scikit-learn works this way, for instance. The trouble with this approach, however, is that this is also likely to boost the predicted anomaly probability for non-anomaly events. You might get much better sensitivity to anomalies but at the expense of specificity and (probably) positive predictive value ("precision" in some circles).

What's another way to boost the sensitivity at the expense of the specificity? Look at the ROC curve and pick the threshold corresponding to a different point, and then use that threshold to bin the predicted probabilities into categories. In fact, changing the prior probability does not change the ROC curve, since ROC curves only depend on ranks and changing the prior corresponds to a rank-preserving transformation.

However, you don't even have to use thresholds at all. You can just predict the probability of an anomaly. Then oversampling, undersampling, or otherwise balancing the categories distorts the predicted probabilities, and there is little point in doing so outside of some special cases that are very-much outside the usual workflow of machine learning.

The utility of predicting probabilities instead of predicting classes is discussed on the blog of Frank Harrell, founding Chair of Biostatistics at Vanderbilt University.

Classification vs. Prediction

Damage Caused by Classification Accuracy and Other Discontinuous Improper Accuracy Scoring Rules

That many so-called "classification" models like logistic regressions and neural networks explicitly predict probability$^{\dagger}$ values and that these probabilities can be evaluated without any reference to a threshold is missed by many practioners of machine learning. Readers of those two Harrell blog articles will see that he finds this frustrating (as do I).

$^{\dagger}$These probabilities might not correspond to the real event probability, the subject of . However, calibrated or not, they are legitimate Bernoulli probability parameters that satisfy the Kolmogorov axioms of probability. Whether or not they are good (calibrated) estimates of the conditional Bernoulli probability parameter is separate.

$\endgroup$
1
  • $\begingroup$ Thank you Dave, I am using the approach you've mentioned. $\endgroup$ Commented Dec 4 at 5:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.