2
$\begingroup$

I have a question regarding a a sinlge-choice problem in my time series course. My question is the following: Is the variance of a process in first differences larger than the variance of the undifferenced series?

I cannot find anything on this either in the course material or on the internet. So my intuition is that differencing is a special case of linear filtering, where the weights do not add up to one (therefore changing the mean and variance). But I am not sure how to check whether the variance does indeed grow larger or not.

I greatly appreciate your responses.

$\endgroup$
2
  • 1
    $\begingroup$ Since you assume nothing about the process (apart from the implicit requirement of second order stationarity needed for "variance" to have any meaning), you should easily be able to construct examples where the direction of the inequality changes. As an example of a variance reduction, consider the series $X_t=X$ for all $t,$ where the differenced series is constantly zero. To create an example with a variance increase, consider the process $X_t=(-1)^t X$ where $X$ is symmetrically distributed. $\endgroup$
    – whuber
    Commented Jun 14, 2022 at 17:10
  • 1
    $\begingroup$ That helps a lot! Thank you! $\endgroup$
    – DLTS
    Commented Jun 14, 2022 at 17:37

1 Answer 1

4
$\begingroup$

For "the" variance of a process $X_t$ to mean anything at all, we must suppose (at a minimum) that the variance of $X_t$ is the same for all $t.$ Let's call this common (finite) variance $\sigma^2.$

Almost always, such a constant-variance assumption goes along with the stronger assumption of second order stationarity, which adds that for any "lag" $s,$ the covariance of $(X_t, X_{t+s})$ is the same for all $t$ (but, of course, may vary with the lag).

The first-differenced process is, by definition, the most recent change

$$(\Delta X)_t = X_t - X_{t-1}.$$

Using the bilinearity of covariance we may compute

$$\begin{aligned} \operatorname{Var}((\Delta X)_t) &= \operatorname{Var}(X_t - X_{t-1}) = \operatorname{Var}(X_t) + \operatorname{Var}(X_{t-1}) - 2 \operatorname{Cov}(X_t, X_{t-1}) \\ &= 2\sigma^2(1 - \operatorname{Cor}(X_t, X_{t-1})). \end{aligned}$$

Writing $\rho$ for the lag-1 correlation that appears in this formula, the question asks us to compare $\sigma^2$ to $2\sigma^2(1-\rho).$ Their ratio (assuming $\sigma^2\ne 0$) is $2(1-\rho),$ which exceeds $1$ when $\rho \lt 1/2$ and otherwise is less than or equal to $1.$ Thus, the variance of the differenced series may be larger or smaller than the original variance, depending on the lag-one correlation.

When $X_t$ is a second-order stationary time series, the variance of its first difference increases when the lag-1 autocorrelation is less than $1/2$ (including all negative values); the variance decreases when the lag-1 autocorrelation is greater than $1/2$ ; and otherwise the variance remains the same when the lag-1 autocorrelation equals $1/2.$

These plots illustrate the phenomenon. The black points and lines graph realizations of the original series $X_t$ while the red lines graph their first differences.

Figure

At the left, the negative correlation means $X_t$ tends to swing back and forth around an average of zero, causing the differences to be larger than $X_t.$ At the right, the positive correlation means each $X_{t+1}$ is close to its predecessor $X_t,$ whence the differences tend to vary less than the original values. The middle shows the boundary case $\rho=1/2$ where the differences tend to have the same sizes as the original values.


This analysis gives us a way to construct examples. The two extremes are the cases $\rho=\pm 1.$ When $\rho=1,$ each $X_{t+1}$ must be an (order-preserving) linear transformation of $X_t,$ so taking these transformations to be the identity, we obtain an example where all the $X_t$ are the same variable. Their common variance $\sigma^2$ is reduced to zero because $\Delta X_t = 0$ for all $t.$

At the other extreme, when $\rho=-1,$ again each $X_{t+1}$ must be an order reversing linear transformation of $X_t,$ of which the simplest is $X_{t+1}=-X_t.$ This leads to the example $X_t = (\pm 1)^t X$ where the series alternates between the value of a random variable $X$ and its negative. Now the differenced series alternates between twice $X$ and its negative, which has four times the original variance.

It is worth noticing that differencing a series with zero lag-1 autocorrelation (such as a series of independent variables) causes the variance to double. This is because each successive difference involves two variables and their variances add.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.