I am working on this problem for my research. The attached time series represents the memory usage of an application over time. As you can imagine, the memory usage steps up randomly every few days. Once in a while, there is a step down (or a spike down) as well (not seen in this figure), but usually the plot keeps growing over a long period of time.
I have applied some of the std. prediction techniques such as Sen's slope, Linear model, Quadratic, 3rd degree polynomial, and even exponential (y_hat = b_0 b_1 ^ t + e_t). Sometimes the prediction is ok, but usually it is sensitive to the duration of the training data and the shape of the plot. In this plot, the dash-dot line separates the training data from test data.
My requirements are:
- Analyze such plots in an automated manner, online, every night.
- Predict the metric for the next 1-2 days with a confidence interval.
How should I go about this?