4
$\begingroup$

I wonder if there are some algorithms that can account for underlying system dynamics over time.

One possible situation can be the following: in a ticket reporting data, a data point arrives when a problem is reported, and a ticket log is created (it contains information on ticket characteristics). These data points are not i.i.d - I can assume they are i.i.d. in a short time frame, but in the long run, system changes (for example, a previous problem got fixed, system got upgraded, etc., which are not quantified or recorded), so it is not appropriate to assume one same distribution for earlier data and current data because they are generated by users living under different system conditions (and there is no information on how often the system changes, plus the change is generally continuous and even different for different aspects). They are probably still independent since tickets are generated by different users, but not quite identically as time evolves.

It seems to me most time series algorithms still assume i.i.d - please correct me if I am wrong - so I am not sure if I can directly make use of them. In addition, I am targeting at a classification problem.

One naive approach I have is to keep updating the model, for example, every $q$ days based on only the most recent information (e.g. within last $p$ days). I am not sure if this is a valid one, but one problem is there are many tuning parameters here such as $p$ and $q$. While this can possibly be done heuristically, I wonder if this can be modeled analytically.

It will be great if I can be pointed to existing algorithms, if any, that already aim to address this kind of scenario, literatures that talk about this or even keywords to search for. Thanks!

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ For the title, the word you want is "evolution". For the question, most time-series models in intro stats/econometrics perhaps assume stationary data, but many models do not. If you have known predictor variables $x$, then standard regression $y(x)$ can be applied in a time series context. If the predictors are not measured, then a standard approach is state-space-models. $\endgroup$
    – GeoMatt22
    Sep 27, 2016 at 21:45
  • $\begingroup$ @GeoMatt22 Thanks! Changed the title. I have predictors that are recording ticket characteristics, but not directly about the system. The problem I did not simply train a classification model on all historical data is that I think model estimates changes over time. So this goes back to retrain the model frequently as mentioned above. $\endgroup$
    – hurrikale
    Sep 27, 2016 at 22:03
  • $\begingroup$ You don't need i.i.d. for time series modeling. It helps when it's the case, but is not strictly necessary for all models. For instance, you could proceed with OLS regression in presence of autocorrelated errors. You may want to make some adjustments to correct a few things that fall off $\endgroup$
    – Aksakal
    Apr 18, 2017 at 13:45

1 Answer 1

Reset to default
2
$\begingroup$

The problem that you are facing is called Concept drift.

The solution you suggested, relearning the model once in a while, will work but it is wasteful since it dismiss most of the data and the insight form the past. Your approach of using only recent data can be considered as coping with the problem in the framework of stream mining and you might find interesting ideas there.

You are correct saying that choosing the learning period parameters is not trivial. The longer it will be you will have more data but pay in your ability to adapt rapidly to a change.

Another approach could be to use domain adaptation methods in order to cope with the change. This way you learn a model on all past event/late past event and once you have a model you adapt it to the recent event.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.