1
$\begingroup$

I have a series of $n$ machines that are going to emit some sensor data. The machines are going to be started at some point, telemetry collected every minute for some time and then stopped. So, instead of one long time-series, I have many short time series. Note that some machines might run for an hour, others for 50 minutes, others for 40 minutes and so on. Within the hour, there will be some seasonal patterns.

Now, I want to fit a time-series model that gives me some 95% confidence bands for each time instant since a machine starts (for the sensor value). Also, tomorrow I will get some new machine I haven't seen before but one that is expected to behave like the $n$ machines I trained on. For each minute and sensor value it produces, I want to estimate the p_value, denoting what the probability of seeing that observation would be if the machine were no different from the $n$ machines I saw in the past.

What time series models can be good fits for this use-case? As a stretch goal, each of the $n$ machines might have some feature vectors associated with them. Is it possible to take the features into account?


Some thoughts: perhaps we can combine the $n$ time series into one long time series and consider the intervals for which the shorter ones don't have values as missing? But then the question becomes what order we should combine them in?


Some examples of the panel data:

https://1drv.ms/x/s!AiY4k2EqE618gakcA0pRg0SAvQl_UA?e=ir93Db

$\endgroup$
2
  • 1
    $\begingroup$ Would it be possible to consider the data as panel data (with unequal numbers of observations for each machine, maybe even some machines give rise to multiple series?) Then there might be some random effects representing some variability between machines? Can you share (a link to) the data, or some data? $\endgroup$ Jul 23, 2020 at 17:47
  • $\begingroup$ @kjetilbhalvorsen - thanks for the pointer to the panel data. Added a link to an excel with some samples. Still looking for a good solution. $\endgroup$
    – ryu576
    Jul 24, 2020 at 7:05

1 Answer 1

2
+50
$\begingroup$

When dealing with large numbers of time series, you need to look at the difference between local time series models and global time series models:

  • Local time series models involve training multiple (usually statistical) models, on for each individual time series. E.g. 100 time series 100 ARIMA models, each with its own p,q,d values.
  • Global models involve training one large ML model on all of the time series. Batching and suitable sampling schemes are used to shuffle through the multiple time series during each training iteration.

For the global model to work though, especially with your new machine cold-start requirement, you definitely need to have the additional feature vectors, not just a a stretch goal, because those features are what will allow the global model to match the new machine to the historical series of similar machines.

What time series models can be good fits for this use-case?

This is a perfect use case for a global deep learning/seq2seq model. Both empirical results and theoretical analysis show that seq2seq models work best with large groups of short time series, while local models work better with small numbers of long time series.

perhaps we can combine the n time series into one long time series and consider the intervals for which the shorter ones don't have values as missing?

Your idea is actually a good one in spirit, because its trying to effectively duplicate the idea behind global time series models. However, it won't work because (a) it would introduce fake patterns and correlations in the data regardless of the order you use to patch the series together. (b) it doesn't solve your new machine cold-start problem.

Now, I want to fit a time-series model that gives me some 95% confidence bands for each time instant since a machine starts (for the sensor value).

If you only need one confidence interval (e.g. 95%) all the time, then just train your global model using pinball loss, with $a=0.95$.

If you need arbitrary confidence intervals, then you will need to forecast full densities, which is also possible using seq2seq and deep learning models.

$\endgroup$
4
  • $\begingroup$ Thanks a lot. For the local models (which might be simpler to implement over a weekend), is there a way to get confidence intervals and p-values for testing observations given the set of ARIMA models? For the deep learning seq-2-seq, haven't used them. Is this a good place to start in your opinion? proceedings.mlr.press/v89/mariet19a/mariet19a.pdf $\endgroup$
    – ryu576
    Jul 25, 2020 at 18:59
  • $\begingroup$ Any decent statistical time series API that you use for local modals will give you the forecast intervals (strictly speaking they are not called confidence intervals) out of the box. But local models by definition can't do cold-starts - I don't know how crucial that requirement is for your use case. $\endgroup$
    – Skander H.
    Jul 25, 2020 at 21:29
  • $\begingroup$ The paper you mention does a good job of exploring the theoretical angle. I recommend this one for the more practical aspects: arxiv.org/pdf/1909.00590.pdf $\endgroup$
    – Skander H.
    Jul 25, 2020 at 21:34
  • $\begingroup$ Ok, will go through. The cold start problem is there, but it isn't so bad since we do have some observations from the past. The machines were running in a different environment then, but this should be a good proxy. And in a few days, we'll have time series observations from the new environment. It doesn't seem trivial how to combine the confidence intervals/ distributions of the predictions from these local models into one global model though. $\endgroup$
    – ryu576
    Jul 25, 2020 at 22:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.