There are two posts on CV about differentiation between short-term and long-term forecasting. e.g. here. There is nothing on CV about combining (incorporating) long-term dynamics into short term forecasting. Is this irrelevant?
I wonder if someone can show/explain what is the statistical procedure of incorporating long term time series dynamics into short term forecasting?
I would like to forecast short-term by incorporating the long term: trend etc. Quite often is the case that short term forecasting is insufficient (short sighted) as in number of cases the time series converges to long term mean. (I understand this assumption is data dependent, but this is type of data I'm dealing with).
Is also the model choice in this case important? So the question is how to combine these two (short/long-term dynamics)?
Actual example (fitted model) within R
would be desired result. For nice answer I'm offering double the current bounty.
EDIT: I'm going to disappoint in term of data and provide data from the forecast
R package, since I think (for my purpose) it is going to be sufficient to answer this question.
Demonstration in R:
library(forecast)
data(gold)
plot(gold)
abline(h=mean(gold, na.rm=T), col="orange3", lwd=2)
lines(fitted(lm(gold[1:800]~index(gold[1:800]))), col="blue3", lwd=2)
abline(v=800, col="red3", lty=2, lwd=3)
I fitted number of models "plainly" to the data (gold[1:800])
and all models would blindly forecast the upward trend. Here is example (picture nr. 2):
plot(forecast(auto.arima(gold[1:800]), h=200))
We divide the sample data up to obs 800
. (what we would observe-"in-sample"). In light of the "long-term" mean value (approx. mean 390) and the last observation value just bellow 500
, I would like to forecast up to step 1000. (index(gold)
) taking into account the long-term mean value
and hence incorporate it into forecast.
Since here we are dealing with steps rather than time, imagine that steps 1:800 is "long-term" (in years) and from 800>
the steps are of small scale, let say "hours". (possible the time representation doesn't matter).
So starting with forecast from the observation 800
forecast next 200 steps by taking into account the "long-term" mean value.