1
$\begingroup$

I'm not that familiar with time series models, but I wanted some help to understand if there is a technique to handle outliers in periods where there are small number of observations.

For instance, if we have a time series data about system performance at hourly levels, it is expected for this data set to have lower number of observations at times of out of working hours. However, if this time series uses average as an aggregation function, we might detect some spikes in some days without that being the expected behavior due to a small number of observations (smaller than the expected behavior for that hour).

Considering I'm not interested in spikes happening in "smaller than expected counts of observations" at a given hour, what kind of technique or model we could use to tune/emphasize outliers on more representative ranges?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ You say the data is hourly? Why there is fewer observations outside of working hours then? Arent the observations coming in every hour? $\endgroup$
    – Jon Nordby
    Commented Jun 27, 2020 at 8:23
  • $\begingroup$ You could train two models. One for working hours, and one for outside-working hours. Or use the same model, but tune the decision rule differently for the two scenarios (when to consider something an anomaly). $\endgroup$
    – Jon Nordby
    Commented Jun 27, 2020 at 8:25

1 Answer 1

Reset to default
0
$\begingroup$

One option is to look at inter-arrival times of the events themselves, rather than aggregated over hours. This gives a more fine-grained representation of the data. This link provides an example from Ted Dunning. It involves some assumptions, but it gives you a feel for how to think about it: (archived) link.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.