I have a time series which has a very strong upward trend for the first half, then very strong downward for the second half and finishes pretty much back where it started. Should I split the data in two for analysis - or can I still account for a trend which is net neutral?

  • 6
    $\begingroup$ There should (or at least could) be some substantive knowledge explaining this strong qualitative behaviour. Similarly, separate analyses for separate parts of a series imply quite different generating processes; that could make sense if you had knowledge that the system became fundamentally different at the turning point. Absent this, it is difficult if not impossible to give a worthwhile technical reply to your question without much more information. $\endgroup$
    – Nick Cox
    Mar 16 '14 at 12:59
  • 3
    $\begingroup$ You mention analysis but it's not clear what you actually want to do. $\endgroup$
    – Wayne
    Mar 16 '14 at 15:31

I think you can actually fit one model that can capture both upward and downward trends and there is no need to split your data (unnecessary) resulting in two models. In the following codes I have consiered an ARIMA model (red line) as well as plynomial regression of degree 2 (blue line). I fitted these model just as an example. Sometimes you actually need to split your data (See Nick's comment).

> x=0:40
> y=-(x-20)^2+rnorm(length(x),0,15)
> plot(x,y)
> library(forecast)
> f=auto.arima(y)
> summary(f)
Series: y 

          ma1     ma2
      -1.4522  0.7282
s.e.   0.1240  0.1156

sigma^2 estimated as 451.2:  log likelihood=-175.89
AIC=357.78   AICc=358.46   BIC=362.77

Training set error measures:
                    ME     RMSE      MAE      MPE     MAPE      MASE
Training set -6.162264 20.71692 16.32124 30.21502 52.33051 0.1559912
> lines(fitted(f),col="red")
> f2=lm(y~x+I(x^2))
> summary(f2)

lm(formula = y ~ x + I(x^2))

    Min      1Q  Median      3Q     Max 
-40.465  -8.068   2.406   9.482  23.517 

              Estimate Std. Error t value Pr(>|t|)    
(Intercept) -401.88742    6.64150  -60.51   <2e-16 ***
x             39.38738    0.76813   51.28   <2e-16 ***
I(x^2)        -0.97488    0.01857  -52.51   <2e-16 ***
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 14.87 on 38 degrees of freedom
Multiple R-squared:  0.9864,    Adjusted R-squared:  0.9857 
F-statistic:  1381 on 2 and 38 DF,  p-value: < 2.2e-16

> lines(fitted(f2),col="blue")

enter image description here


Such time series are usually hard to model using ARIMA. But I suggest you plot the ACf and PACF of the series and see. It may require differencing. You can also test to see if the variance is constant; if it is not you transform. Once the data is stationary you can make sense out of it.

As Nick has said splitting the series will generate two different processes and more information is needed.

The series might appear to be oscillating: if that's the case the best thing to do is to try spectral analysis.

  • 4
    $\begingroup$ Thanks for the mention, but you have my point backwards: if the processes generating the series are different, then separate models might make sense; it's processes that generate data, not vice versa, as I use terms. Also, the implication that you need to make series stationary to make sense of them does not seems to me to reflect best current thinking on time series. $\endgroup$
    – Nick Cox
    Mar 16 '14 at 13:35
  • $\begingroup$ Thanks for the correction. Making the series stationary before analyzing might no be the best way, but it makes it easier to analyze that way. $\endgroup$
    – frisbee
    Mar 16 '14 at 13:55

I have had significant success in empirically identifying trend changes using AUTOBOX http://www.autobox.com/cms ,a commercially available piece of software that I have helped develop. You might want to look at stochastic vs deterministic trend/seasonality in time series forecasting for an interesting discussion.

A time trend model (deterministic in form) is as follows y(t)=a+bx1+cx2
etc where x1=1,2,3,4....t and x2=0,0,0,0,0,1,2,3,4 thus one trend applies to observations 1−t and a second trend applies to observations 6 to t.

AUTOBOX's automatic empirical procedures have been very successful BUT like all "new science" or "advanced innovative procedures" it needs to be constantly aggressively challenged. Test all things but hold fast to what you know to be true ! Please post your data or send it to me privately and I will use the data and report the results to the group. If you are skittish about releasing the data simply code it by adding/subtracting a constant. The procedures used to identify the number of trends and the length of each trend is based partially on the work of Tsay http://www.unc.edu/~jbhill/tsay.pdf. All of these advanced procedures are not currently available in the free software market.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.