I want to fit an ARIMA model for forecasting on a quarterly basis, but my data is a rolling year, updated quarterly, how can I use this most effectively? I'm really interested in the best estimate of the future quarterly value I can get.
On the one hand I'm concerned about the "artificial" serial correlation due to the overlap in quarters, also that the averaging across the quarters is smoothing the series relative to its quarterly counterpart. On the other, throwing out the data seems like a waste of information. Just to clarify, the rolling years go: Jan 2004-Dec 2004, Apr 2004-Mar 2005 etc.
A few thoughts on this so far:
- Differencing the series removes the overlapping quarters and produces a series of the annual change from, say, Q1 2015 to Q1 2016, Q2 2015 to Q2 2016 etc.
- An assumption could be made about the value of one quarter, and then the quarterly series could be backed out of the rolling series, e.g. T1 = Q1 + Q2 + Q3 + Q4, and T2 = Q2 + Q3 + Q4 + Q5, then T2 - T1 = Q5 - Q1, so assume Q5 = X, and solve for Q1 or vice versa. However the entire series produced will be very sensitive to this assumption.
Take, as an example, employment data from the UK Annual Population Survey, here I've got employment and unemployment data for men and women in London, aged between 16 and 24. The plots here are male employment, but you can run the others in the code provided. (I've just put up one comparison to keep things clear.)
I forecast the data below using the rolling annual series and again using only the complete years from Jan-Dec with no overlapping quarters (so, an annual series). In the latter case there's not much to fit a model with, and the forecasts are clearly a lot simpler. It seems preferable to at least use the additional information and be careful about units (that what's being forecasted is the rolling year for the next period, not the next quarter). However both are really more like annual forecasts as the quarterly volatility is being smoothed out.
library(tidyverse) library(httr) library(stringr) library(zoo) library(forecast) library(gridExtra) # get the data df = GET("http://www.nomisweb.co.uk/api/v01/dataset/NM_170_1.data.csv?geography=2013265927&c_sex=1,2&c_age=22&c_ecopuk11=2,12&measure=1&measures=20100,20701&select=date_name,geography_name,geography_code,c_sex_name,c_age_name,c_ecopuk11_name,measure_name,measures_name,obs_value,obs_status_name") employment = df %>% content("text") %>% read_csv() # build a series using the rolling years df2 = employment %>% mutate(date_order = ((str_split(employment$DATE_NAME, "-")) %>% map(~as.yearqtr(., format = "%b %Y")))) %>% select(date_order, date = DATE_NAME, geography = GEOGRAPHY_NAME, sex = C_SEX_NAME, age = C_AGE_NAME, status = C_ECOPUK11_NAME, measure = MEASURES_NAME, value = OBS_VALUE) %>% filter(measure == "Value") %>% mutate(chars = str_c(sex, age, status, sep = "")) %>% select(-geography, -sex, -age, -status, -measure) %>% spread(chars, value) # drop all of the quarters in between and use only Jan-Dec years df3 = df2 %>% filter(str_detect(date,'Jan')) # plot plot1 <- auto.arima(ts(df2$`FemalesAge 16 to 24Employed`,frequency = 4)) %>% forecast(h = 10) %>% autoplot plot2 <- auto.arima(ts(df2$`MalesAge 16 to 24Employed`,frequency = 4)) %>% forecast(h = 10) %>% autoplot plot3 <- auto.arima(ts(df2$`FemalesAge 16 to 24Unemployed`,frequency = 4)) %>% forecast(h = 10) %>% autoplot plot4 <- auto.arima(ts(df2$`MalesAge 16 to 24Unemployed`,frequency = 4)) %>% forecast(h = 10) %>% autoplot grid.arrange(plot1, plot2, plot3, plot4, ncol=2) reducedplot1 <- auto.arima(df3$`FemalesAge 16 to 24Employed`) %>% forecast(h = 10) %>% autoplot reducedplot2 <- auto.arima(df3$`MalesAge 16 to 24Employed`) %>% forecast(h = 10) %>% autoplot reducedplot3 <- auto.arima(df3$`FemalesAge 16 to 24Unemployed`) %>% forecast(h = 10) %>% autoplot reducedplot4 <- auto.arima(df3$`MalesAge 16 to 24Unemployed`) %>% forecast(h = 10) %>% autoplot grid.arrange(reducedplot1, reducedplot2, reducedplot3, reducedplot4, ncol=2)