Are there ML algorithms for data whose probability distribution changes over time?

Question

It seems ML algorithms are specialized for cases in which the population distribution is fixed. Cross-validation also wouldn't work well if the distribution would change over time. However, it is not uncommon to encounter stochastic processes whose probability distribution changes with time because the underlying dynamics are not fixed.

The problem I am considering is this:

1. The data generating probability distribution changes over time. (We may assume the distribution family to be fixed with time-varying parameters.)

2. The new data generating process might be identical to one of the data generating processes encountered before.

The learning algorithm should be able to recognize the new data generating process when it's changed and it should be able to train with the new data.

At the same time, to handle the second situation, the learning algorithm should be able to remember the historical data distributions. And when it recognizes the new generating process, the algorithm should be able to retrieve the historical model and predict with it.

But the recognizing process would not be perfect and it would construct the current model as an ensemble of learners from the historical model.

This seems to be quite a complicated learning algorithm and I don't know if this is even possible. Is there any research in this direction?

Answer 1

Not sure if this is what you're looking for, but there are of course decades of research on time-varying models in the dynamical systems community.

As a simple example [we will look at extensions below], let's say you're interested in a time-varying linear model $$ y(k) = x(k)^T \theta(k) + \varepsilon(k).$$ Assuming a simple random-walk model for the parameters $\theta(k)$, the joint model can be written as the state-space model $$ \theta(k) = I \cdot \theta(k-1) + \mu(k), \quad \mu(k)\sim\mathcal{N}(0,\Sigma_\mu) \\ y(k) = x(k)^T \theta(k) + \varepsilon(k), \quad \varepsilon(k)\sim\mathcal{N}(0,\Sigma_\varepsilon).$$ The MMSE estimator for this model is the classical, linear Kalman filter / smoother. The velocity of the parameter adaptation can be adjusted by means of the assumed process noise covariance $\Sigma_\mu$.

• This type of models appears to have been somewhat popularized in ML / statistics in recent years under the name of "Bayesian structural time series models" [1],[2]
• If you're interested in more complex models of the dynamics of the parameter changes, you can exchange the identity matrix for more interesting models which, e.g., describe periodic dynamics or trends.
• If you're interested in systems with rapid change points, you can use other models for the process noise, for instance, a NUV prior. (Inference is then performed by means of a joint Kalman filter + expectation maximization algorithm.)
• If you're interested in nonlinear time-varying models, you can do the same thing, except that the observation equation then is nonlinear and you have to use some nonlinear Kalman filter / smoother variant.[3][4]
• This kind of inference algorithm always remembers all previously observed data points and does not "forget". (Except if you explicitly model it that way.)