1
$\begingroup$

There are many tutorials/packages in Python to detect anomalies in time-series given that the time-series is numerical.

Currently, I have a time-series that is categorical, i.e. the time-series data said that, at time XXX the event AAA occurred.

I want to detect anomalies for this data. For instance, if too many events BBB occured in a short period of time...

Could you give me some starting point (if it is in Python, it's great).

Many thanks

$\endgroup$
1
  • $\begingroup$ XXX is still a real timestamp right? I that case I think the appropriate terminology is that the time-series data is irregular / unevenly spaced in time? If it is very sparse, it might be referred to as a sequence of events. $\endgroup$
    – Jon Nordby
    Mar 22, 2023 at 21:41

1 Answer 1

0
$\begingroup$

What you have there seems to be an uneven/unequally/irregularly spaced time series, consisting of discrete (in time) events.

The most straightforward approach for anomalies in number-of-events would be to transform the data into a regular time-series, and use a general-purpose anomaly detection model.

import pandas

events = pandas.DataFrame.from_records([

# B normally happens once per day
{ 'time': '2023-03-22T18:22:00', 'event': 'B' },
{ 'time': '2023-03-23T18:22:00', 'event': 'B' },
{ 'time': '2023-03-24T18:22:00', 'event': 'B' },

# skipped a day
#{ 'time': '2023-03-25T18:22', 'event': 'B' },

# many times in one day
{ 'time': '2023-03-26T16:22:00', 'event': 'B' },
{ 'time': '2023-03-26T18:22:00', 'event': 'B' },
{ 'time': '2023-03-26T20:22:00', 'event': 'B' },

# A once per week
{ 'time': '2023-03-26T20:22:00', 'event': 'A' },

])
events['time'] = pandas.to_datetime(events['time'])

print(events)

# Transform to regular time-series with counts
time_bins = '1d'
regular = events.set_index('time').groupby('event').resample(time_bins).count().rename(columns={'event': 'count'})
print(regular)

# Transform into one column per event
data = regular.reset_index().pivot(index='time', columns='event', values='count').add_suffix('_count')
data = data.fillna(0.0)
print(data)

# Can do further feature engineering here, like split time into weekday/time-of-day
# and then pass to a standard Anomaly Detection method, such as IsolationForest

Should print

                 time event
0 2023-03-22 18:22:00     B
1 2023-03-23 18:22:00     B
2 2023-03-24 18:22:00     B
3 2023-03-26 16:22:00     B
4 2023-03-26 18:22:00     B
5 2023-03-26 20:22:00     B
6 2023-03-26 20:22:00     A

                  count
event time             
A     2023-03-26      1
B     2023-03-22      1
      2023-03-23      1
      2023-03-24      1
      2023-03-25      0
      2023-03-26      3

event       A_count  B_count
time                        
2023-03-22      0.0      1.0
2023-03-23      0.0      1.0
2023-03-24      0.0      1.0
2023-03-25      0.0      0.0
2023-03-26      1.0      3.0
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.