lag dependent variable in time series - production phase

Question

I'm having a problem in understanding how to overcome this problem.

I'm aware that in order to apply various machine learning algorithm i can transoform a time series problem which is unsupervised into a supervised problem by providing lags of dependent variable https://machinelearningmastery.com/convert-time-series-supervised-learning-problem-python/.

That's ok.

But how I can produce this lagged variable in a real world scenario, when I'm trying to put my model into production? I'll make an example:

Let's suppose my model is a feed forward NN. I have a time series recorded on a daily basis. As a regressor I included the dependent variable lagged 30 days and other predictors extract from date( day of month, month, quarter, holiday etc etc). Now I'm at the last day of July and I want to predict my time series for the next 30 days. For example At time t+28 I should provide my model with lagged variable til t-30. But I don't have any data for the first 28 day of August. How should I proceed? I should take one step forecast into my model and using it as a dependent variable? I think that in this way i will introduce a substantial error in my model, if my predictions are not good enough.

Related to this issue: If I provide feature like a rolling window 30 days statistics ( let's say mean, std, or kurtosis) of my dependent variable, how should I compute this statitics? At time t+28, i should compute the mean of my data using 30 previous day, which I don't have.

Thanks

Answer 1

If you are correctly incorporating day-of-the-week, day-of-the month, month, quarter, lead & contemporaneous & lag holiday effects and step/level shifts and local time trends AND accommodating one-time pulses and possible changes in day-of-the-week effects you probably only need (if any) short term arima structure. If your model requires some 30 lags , I would question the modelling procedure.

In effect a 30 day forecast is made by bootstrapping predictions based upon the idea that predictions are a substitute for future values due to an expected error of 0.. This can be approached via SARMAX models or as it also known as a Transfer Function models where attention/remedial structure has been given to possibly both time-varying parameters and time-varying error variance. In this way your secondary questions/concerns are voided.

It appears that your approach requires you to pre-fill the future values/expected values of the dependent variable which of course you don't have unless you make 30 runs and then use the 30 expectations as "actuals". This is grossly inefficient and requires programming support and NO routine ability to place confidence limits around expectations which should be part and parcel of any forecasting application.