Time Series: making use of extraneous data

Question

Background: My background is in statistics, but I have very little experience in working with time series data, so please go easy on me if I'm not making sense :)

Case Study: Let's suppose I would like to forecast honey output in the next quarter. I have data about historical honey outputs as well as a few other variables such as weather conditions, the number of active beehives and so on (note: these are not going to be known for the period I will be forecasting).

Problem: This seems like a classical time series problem, however, most time series methods either (1) cannot make use of extraneous regressors or (2) require these regressors to be known for the period we are trying to forecast.

---

Question 1: Are there time series methods out there that can indeed make effective use of extraneous data that will not be known for the period we are forecasting? If so, why are they not popular? Is it because the lagged response variable tends to give us sufficient amount of information anyway?

Question 2: Can this problem be reformulated as a general regression on lagged inputs (after removing seasonality and trend)? If so, are there any reasons I may still choose to stick to the time series approach?

Question 3: Based on the description of the case study, is there any particular method that you could recommend? (e.g. Bagged ETS, TBATS, NNs, ARIMA. etc)

Resources: Lastly, if you can recommend any particular resources or libraries (preferably in R), please do let me know.

Many thanks!

Answer 1

1. If your focal series is driven strongly by explanatory variables, then you should include them. If you don't know them ahead of time, try to forecast them themselves. Or set them, if they are intervention variables.

   For instance, I forecast retail sales. These depend heavily on promotions and price changes. The user will need to supply future promotions and price changes, otherwise the forecasts will be bad. No way around this.

   Note that if you need to forecast explanatory variables, then these forecasts carry uncertainty. You should really carry this uncertainty forward to your final forecasts, e.g., by simulating.

2. Sure! In fact, an autoregressive (AR) model is not much more than a regression on the lagged values of the time series itself.

   Be careful if you remove trend and seasonality first and then regress on explanatory variables. You will likely end up with biased estimates. Consider trying it the other way around: first regress on your explanatory variables, then model the residuals using time series algorithms. For instance, you could do a regression with ARIMA errors. (This is not ARIMAX. Take a look at Rob Hyndman's blog post on "The ARIMAX model muddle".)

3. See point 2 above. Regression with ARIMA errors would be a good place to start, using the forecast package in R. Simply feed your explanatory variables into the xreg parameter during fitting. (And per point 1 above, you will need to also provide them in forecasting. Forecast the explanatory variables themselves if necessary.)

   ETS can deal with explanatory variables through its state space formulation, though I think the standard forecast package may not yet support this - there are some specialized packages from the Lancaster forecasters which should be helpful. With NNs, you will need to specify the lags yourself, the forecast package's nnetar() function does not automatically model seasonality.

   If you suspect that the causal arrow does not necessarily run from one variable to the other, but possibly also the other way around, potentially with lags, you may want to look at vector autoregression (VAR). This is often a useful assumption in macroeconomic data, with GDP, output and job market data all interrelated.