6
$\begingroup$

Apart from detecting trend from a time series plot, how do you test for its presence before removing the trend using moving average?

I fitted a mathematical trend to the data and the slope was approximately zero, does that mean there is no trend?

$\endgroup$
  • $\begingroup$ Answers depend very much on the structure in the data, both in terms of what the trend consists of and how the dependence over time comes in. There are a number of tests that might be suitable (or might not!); an example would be the Mann-Kendall test for trend. $\endgroup$ – Glen_b -Reinstate Monica Feb 7 '14 at 2:07
4
$\begingroup$

You make assumptions about the behaviour of your trend. Without assumptions you cannot hope to ever test data.

Detrending a time series is effectively modelling a time series.

The most common (and often violated) assumptions you first need to make when dealing with time series are the following.

  1. $E|X_t|^2 < \infty, \quad \forall t \in \mathbb N$
  2. $EX_t = \mu, \quad \forall t \in \mathbb N$
  3. $\gamma(s, r) = \gamma(s + t, r + t), \quad \forall r,s,t \in \mathbb N$

The $\gamma$ in item 3 refers to the correlation function taking $s,r$ indices of the time series as arguments.

Time series that comply are referred to as weakly stationary.

Your data sounds like it already complies to 2. If your data doesn't get "noisier" as time go forward then it also complies to 1.

Often the first step in modelling time series is transforming your data to approximately meet these assumptions, where possible. A simple means of meeting items 1,2, that works some of the time, is to difference the data (repeatedly), wherein you subtract each element of the time-series from the element preceding it. Other options include fitting polynomial regression to the data or performing regression on concatenated windows of a cyclical trends.

What you're hopefully left with is a series that approximately meets these assumptions. There's still plenty of trend to be devoured by your model.

Here's a function to compute $\gamma$.

import numpy as np
def autocorrelation(x):
    assert(len(x.shape) == 1)
    n = x.shape[0]
    x -= x.mean()
    trans = np.fft.fft(x, n=n * 2)
    acf = np.fft.ifft(trans * np.conjugate(trans))[:l]
    acf /= acf[0]
    return np.real(acf)

Now let's look at realizations of some stationary data.

import numpy as np
# Make data that isn't autocorrelated
x_not = np.random.normal(0, 1, 1000)

No trend

Autocorrelation function of no trend

# Make autocorrelated data
z = np.random.normal(0, 1, 1000)
x_auto = z[1:] - .8 * z[:-1]

Has trend

Autocorrelation of a trend

If you want an explicit test then $\frac{1.96}{\sqrt n}$, where n is the length of the time-series, is the 95% confidence bounds on null autocorrelation.

You can get pretty wild with time series modelling, but this is the linear regression analog of time series. The collection of these tricks form the guts of what is known as (S)ARIMA.

$\endgroup$
  • $\begingroup$ Maybe a matter of preference, but to be mathematically unambiguous, I would say that your conditions are those for weak stationarity $\endgroup$ – ekvall Feb 7 '14 at 4:57
  • $\begingroup$ You're correct. :) $\endgroup$ – Jessica Collins Feb 7 '14 at 13:30
1
$\begingroup$

In time series econometrics, an important task is to determine the most appropriate form of the trend in the data, not merely whether a trend exists.

There are two common trend removal procedures: taking say the first difference and performing the time-trend regression (or a non-parametric alternative e.g. moving averages). The former is appropriate for I(1) (read integrated of order one) time series and the latter is appropriate for trend stationary I(0) time series. Unit root tests can be used to determine the nature of the trend (stochastic or deterministic), which will suggests the appropriate way to remove it.

As stated, your question is limited to non-parametric estimation of the deterministic component of a trend-stationary process. It is quite often, in e.g. business and economics, that the observed trend is stochastic in nature. Unit root tests (and corresponding stationarity tests) are tools for determining the presence of a stochastic trend in an observed series. Tests like Phillips-Perron test can accommodate models with a fitted drift and a time trend so they may be used to discriminate between the unit root non-stationarity (stochastic trend) and stationary about a deterministic trend (of a non-stationary process).

There are many good sources for further reading, but probably take a look at this theory and this practice first.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.