0
$\begingroup$

I am trying to predict an index using Google Trend Data. I try to orientate myself by this paper. In this paper the authors use the three variables: Sales, Index and SearchFrequency to forecast the Index(he also forecast the sales but in my case its not necessary). Already here a question arise for me:

(1) In this paper the authors use a "seasonal autoregressive (AR) model" for multivariate data. I thought that AR models can only be used for univariate data. So I am wondering if they used an AR model for every time series separately and put the outputs into a linear regression model? I feel stupid asking that question but that's how I can explain it to myself.

Back to my case: These are my two time series:

Search Frequency:freq <- ts(frequency, start=2004, frequency = 4)(blue-line)

Index:index <- ts(index, start=2004, frequency = 4) (red-line) enter image description here

Both time series are quarterly data points. As you can see in the graph of "SearchFrequency" there is a slight seasonality in the fourth quarter. Because both time series are non-stationary my first step was to transfer them into stationary time series.

(2) Do both time series have to be stationary?

I used autoplot(diff(freq)) and autoplot(diff(diff(index)))

(3) sometimes I also read about taking the log instead of differencing. After log however my time series still weren't stationary so I decided to take the differencing, but it would be still interesting to know when to use what.

after differencing the time series look like this:

Search Frequency: Search Frequency Index: enter image description here

Search frequency for me seems to be stationary with high variance now and index seems also to be stationary but I had to difference twice because after the first time there was still a trend.

To determine appropriate values for p,q I used the ACF and PACF plot:

For Index: enter image description here enter image description here (4) Looking at the ACF Plot we see two significant spikes and in PACF we see three spikes. does that mean that I should use p=2 and q=3?

For Frequency: enter image description here enter image description here

(5) Looking at the ACF and PACF plots of Frequency its getting really confusing for me. The ACF shows a sinusoidal pattern with a lot of spikes and the PACF has 5 spikes if we ignore the spike at lag 7. Which information can I get out of this?

(6) what is the next step? Should I do the auto.arima() with both time series and try to compare them to arima() with the p,d,q I got out of the ACF and PACF plots, compare them and choose the model which has the lowest AIC value?

I am really grateful for any kind of help. I worked with "forecasting. principles and practice" by Rob Hyndman which helped me already a lot.

EDIT:

  frequency:
3
2
5
3
3
4
6
5
8
6
5
4
4
7
7
5
9
8
9
8
9
11
11
11
15
16
19
12
26
30
29
24
32
36
38
28
39
45
48
39
52
55
58
44
65
68
69
59
70
71
75
58
77
79
77
71
90
88
94
75
98
90

index:
83.9
82.8
81.9
80.6
84.2
82.7
84.4
82
83.2
83.5
82.3
83
80
81.7
81.6
81.6
82.8
83.1
81.6
81.8
81.8
83.1
82.9
84.2
83
84.3
84.3
83.8
86
87.1
86.7
87.3
87.9
89.1
90.4
91.6
91.3
93.1
92.9
93
93.8
95.7
96.2
96.2
97.8
99.9
100.4
101.8
103.9
106.9
108.8
110.4
110.9
113.1
115
117.3
118.3
120.6
123.1
124.6
124.4
126.9
$\endgroup$
1
$\begingroup$

Please read https://autobox.com/pdfs/regvsbox-old.pdf to get an idea of causal analysis in time series. Essentially one pre-whitens the data in order to be able to use standard methodologies to IDENTIFY a possible model.

http://www.math.cts.nthu.edu.tw/download.php?filename=569_fe0ff1a2.pdf&dir=publish&title=Ruey+S.+Tsay-Lec1 will also be of some help.

If you post your data in a csv vile I will try and help further.

EDIT AFTER RECEIPT OF DATA:

I took 62 quarterly values fro the Y variable (freq) and the X variable INDEX into AUTOBOX , a piece of software that I have helped to develop, It treats the problem in an holistic manner incorporating differencing when necessary , error variance components to stabilize the model's residual variance , outliers/pulses/season pulses as needed. Sometimes software has features that are unneeded for a particular problem ... sometimes (this case) the data requires a sequence of complex modelling steps that are non-trivial. Your particular data set is non-trivial There is an R version of AUTOBOX .

Here is the plot of the two seriesenter image description here visually suggesting some linkage. Mode identification required using a filter on both Y and X of the following form enter image description here yielding cross-correlations that can be interpreted to suggest the form of the Transfer Function model.

Here is the first iteration .

enter image description here

The residuals from this model suggested possible improvements incorporating an ar(1) augmentation and 2 seasonal pulses suggesting latent deterministic seasonal.

An examination of the model's residuals suggested a deterministic change point in the model's error variance using the TSAY test for constant error variance here [enter image description here] . While it is true that the model error variance is larger at higher levels of Y (suggesting the need for a log transform of Y), it is more trueer that the model's error variance changed/increased deterministically between time region 1-26 versus time region 27-64 4. This is visually obvious when one looks at a plot of Y by itself enter image description here

The final model is here enter image description here with acf/pacf of the model's residuals here enter image description here

enter image description here

The forecast for Y depends on forecasts for X which are shown here based upon the AROMA model for X

enter image description here

leading to a forecast table for Y and an accompanying graph enter image description here

The Actual/Fit and Forecast graph is here enter image description here

$\endgroup$
2
  • $\begingroup$ Thank you for your advanced answer! I have a question: The last graph shows the forecast for Y(freq) right? Did you forecast Y because Y is dependent on X and not the other way around? I mean is it not possible to Forecast X with Y? Or does the Forecast of M_index already contain the data of Y? $\endgroup$ – sense Jan 23 '20 at 8:21
  • $\begingroup$ I forecasted Y using X and it's lags and the past of Y and I used the two significant seasonal pulses (4th qtr and 1st qtr). A totally different model would apply if you reversed the dependency. If you wish to accept my answer and open up a second question where Index was the dependent variable that would be fine. $\endgroup$ – IrishStat Jan 23 '20 at 12:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.