Intuition behind the first, second, third order differencing. When to use them?

Question

I am working with time series data and out of curiosity took the first, second and third order differences of my data. However, I am not quite sure when they are used?

I also wonder about the intuition behind differencing. First order is pretty clear; it is the difference.

$$\begin{equation} \begin{aligned} \Delta X_t &= X_t - X_{t-1}. \end{aligned} \end{equation}$$

The second one as described here is the curvature of the series at a given point in time.

$$\begin{equation} \begin{aligned} \Delta^2 X_t = \Delta (\Delta X_t) &= \Delta (X_t-X_{t-1}) \\[6pt] &= \Delta X_t - \Delta X_{t-1} \\[6pt] &= (X_t-X_{t-1})-(X_{t-1}-X_{t-2}) \\[6pt] &= X_t - 2X_{t-1} + X_{t-2}. \\[6pt] \end{aligned} \end{equation}$$

But I don't know actually what the third difference means and when to use first, second and third differences.

$$\begin{equation} \begin{aligned} \Delta^3 X_t = \Delta (\Delta^2 X_t) &= \Delta (X_t - 2X_{t-1} + X_{t-2}) \\[6pt] &= \Delta X_t - 2\Delta X_{t-1} + \Delta X_{t-2} \\[6pt] &= (X_t - X_{t-1}) - 2(X_{t-1} - X_{t-2}) + (X_{t-2} - X_{t-3}) \\[6pt] &= X_t - 3X_{t-1} + 3X_{t-2} - X_{t-3}. \\[6pt] \end{aligned} \end{equation}$$

Answer 1

Differencing is mainly appropriate when the time series $\{x_t\}$ contains one or more unit roots, i.e. is integrated of order $d=1,2,...$.

I(1): In case of a single unit root, $\{x_t\}$ is I(1); it is a cumulative sum of an I(0) time series, $x_t=\sum_{\tau=1}^t u_\tau$ where $\{u_\tau\}$ is I(0). An example of an I(1) series could be a logarithmic stock price $\{\ln(P_t)\}=\{p_t\}$ where the logarithmic returns $\{\ln(P_t/P_{t-1})\}=\{p_t-p_{t-1}\}$ are approximately I(0). Taking the first difference of an I(1) series allows us to work with the I(0) series, and that is convenient as we have all kinds of useful statistical results derived for stationary series*.

I(2): In case of two unit roots, the first-differenced time series $\{\Delta x_t\}=\{x_t- x_{t-1}\}$ is a cumulative sum of an I(0) time series. In other words, the original series is a "twice-cumulative sum" or "second-order cumulative sum" of an I(0) time series, $x_t=\sum_{\tau=1}^t\left(\sum_{s=1}^\tau v_s\right)$ where $\{v_s\}$ is I(0). Some economists consider the consumer price index to be approximated by I(2) which implies that the price increments are I(1). Indeed, price increments show some persistence, but how good of an approximation I(1) is to them is debatable.

I(3): I am not aware of any I(3) processes in the real world.

If you difference an I($d$) series more than $d$ times, you encounter the problem of overdifferencing. It increases the error variance of the model relative to a model with an appropriate order of differencing (which is $d$) and creates a moving-average component with a negative unit root. That is why differencing should be done with care and not overdone.

*An I(0) time series is quite similar to a stationary one, though there are some subtle differences.