Machine Learning techniques for uneven series of events

Question

I don't know the correct terminology and so long everything I typed into google lead me to some version of time series modeling where all time series had the same number of points in a given time window. So i'll try to explicitly describe the problem and hopefully You can help me out with proper terminology.

My case have a lot of discrete events for each customer. Some customers produce tons of events every day, some may produce an event once a month or even less often (and of course there is a distribution of everything in between).

Those events can happen on short time scale and at the same time there can be a huge gap between events - You can have a 100 events within a minute than few days of nothing than one event than few months of nothing (this example is rather extreme).

Usually response variable we are predicting is something that encapsulates longer trends of interaction with our products - like propensity to buy new product.

We are currently aggregating those events into monthly/yearly/quarterly/etc features + combine with all other features we have about customers and do most ML projects on resulting tabular dataset. Things like predicting some properties of customer, churn prediction, propensity to buy, customer segmentation, etc...

With every new project there is quite an overhead on researching and creating new aggregates based on those events because every time we tackle new problem new features emerge from them that where either considered not related or was not present in the portion of the dataset related to given project.

We are looking into a ways to use raw data of those events instead of aggregates - we can assign a fixed size vector of "raw features" (things we can describe about the event, who, what, where, how, etc) to each event and the goal is to let NN's to learn the right temporal combinations of those features for a given problem.

One of the approaches to building ML system on data of varying length is to use RNN's. We could feed stream of those vectors ordered by the order in which they appeared for a time window of some size and model our target based on those.

Are there alternative approaches? Is the RNN approach even suitable for this type of problem?

What kind of terminology best describes this kind of problem that I can use to further my research into the topic?

Answer 1

The problem you are referring to can be seen as a particular case of Functional Data Analysis (FDA). An excellent review of the topic can be found in [1]. Here is a quote concerning your problem :

« Sparse functional data arise in longitudinal studies where subjects are measured at different time points... »

This suggest that you may find some answers by looking for Sparse Functional Data in the litterature.

Using RNNs is a natural choice for time series analysis. The LSTMs may offer better performance compared to other RNNs when dealing with anytime events as it seeks to maintain an hidden state, which will also help coping with sparsity of the events. LSTMs also have the advantage that you can directly use the data without any aggregation.

Temporal Convolution Networks are a good alternative if you are willing to accept a certain degree of aggregation.

[1] Wang, J. L., Chiou, J. M., & Müller, H. G. (2016). Functional data analysis. Annual Review of Statistics and Its Application, 3, 257-295.