I am working on this problem for my research. The attached time series represents the memory usage of an application over time. As you can imagine, the memory usage steps up randomly every few days. Once in a while, there is a step down (or a spike down) as well (not seen in this figure), but usually the plot keeps growing over a long period of time.

I have applied some of the std. prediction techniques such as Sen's slope, Linear model, Quadratic, 3rd degree polynomial, and even exponential (y_hat = b_0 b_1 ^ t + e_t). Sometimes the prediction is ok, but usually it is sensitive to the duration of the training data and the shape of the plot. In this plot, the dash-dot line separates the training data from test data.

My requirements are:

  1. Analyze such plots in an automated manner, online, every night.
  2. Predict the metric for the next 1-2 days with a confidence interval.

How should I go about this?

enter image description here


Time series data is a collection of transactions for a specific bucket of time. It is often incorrectly/inefficiently analyzed as a cumulative. For example total ticket sales for a movie start at zero and then increase. If you analyze the actual new tickets sales per day or per week you can then get a useful model. Daily Forecasts can always be presented for the cumulative while improved understanding/predictability can be found at the actual observed data level. I suggest that you start to examine the data in the dis-aggregate. If you wish you can post your data in a csv column format and I will address that for you.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.