Is it a valid claim, that by differencing a time series, it loses its memory, and as a result its predictive power?

Question

Marcos Lopez de Prado seems to be a well known and renowned machine learning expert in the field of finance. I am very far from his level, as have not yet finished my PhD in economics, and only have an applied level statistical knowledge. I have encountered a much cited paper of Lopez de Prado. I can not say I completely understand all the mathematical parts. But there are some claims in the paper, which seem to completely contradict things, which I have learned on statistics and economics thus far, or at least seem to be illogical for me. For a specific example, the paper under the section Pitfall #4 and Solution #4 suggests, that by differencing time series to make them stationary for classical statistical models (ARIMA etc.) removes the memory of the series and thus makes them lose the predictive power:

The conclusion is that, for decades, most empirical studies have worked with series where memory has been unnecessarily wiped-out. The reason this is a dangerous practice is that fitting a memory-less series will likely lead to a spurious pattern, a false discovery. Incidentally, this over-differentiation of time series may explain why the Efficient Markets Hypothesis is still so prevalent among academic circles: Without memory, series will not be predictive, and researchers may draw the false conclusion that markets are unpredictable.

In economics there really is a simplified theoretical model on equity returns, which posits, that it is a memory-less white noise, and the prices (the integrated returns) follow a random walk. But from an empirical perspective, as far as I understand the memory-less attribute of returns pertains only to the individual data points themselves, not the series as a whole. A differentiated series still should have a "collective" memory put together, and it has almost the same information as the integrated version, only lacks a constant value. So it should have the same predictive power as well, should it not? Or it is me, who has a lack of understanding?

Answer 1

Since my similar question was flagged as duplicate (good debate in the comments!), I came across Simon Kuttruf's explanation on Medium:

for integer orders of differencing only a (small) finite set of past values is reflected in the resulting differenced series: the preceding value in first order differencing, two preceding values for second order differencing etc. While for fractional orders of differencing, all coefficients take on (asymptotically small) non-zero values and so past values get mixed into the differenced series, up to some chosen cutoff. This phenomenon is here referred to as ‘long memory’ (or ‘wipe-out of memory’ resp).

If the fraction $d = .5$, then the first four coefficients (check my calculations) according to the recursive formula (see Simon's article)

$$ w_k = -w_{k - 1} \left( \frac{d - k + 1}{k} \right), $$ the first four values would be 1, -.5, -1/8, and -1/16, leading to the transformation

$$z_t = y_t - \frac{1}{2} y_{t - 1} - \frac{1}{8} y_{t - 2} - \frac{1}{16} y_{t - 3} + \ldots$$

According to the terminology, the "memory is preserved" because the coefficients are positive for infinitely many past values of $y_{t-k}$.

The one example I can think of where memory is clearly lost by a first difference but without any apparent consequences is a true random walk. Where $\epsilon_i$ are i.i.d. $N(0, 1)$ random variables, let $$ \begin{align} y_1 &= \epsilon_1, \\ y_2 &= \epsilon_1 +\epsilon_2, \\ y_3 &= \epsilon_1 + \epsilon_2 + \epsilon_3, \\ y_4 &= \epsilon_1 +\epsilon_2 + \epsilon_3 +\epsilon_4. \end{align} $$ Then the first difference $y_4 - y_3 = \epsilon_4$ losses all of $y_4$'s information about $\epsilon_1, \ldots, \epsilon_3$, whereas the fractional differencing transformation would contain a bit of all of the epsilons. Not that they'd help you to forecast.

Another maybe more profound example where "memory" matters is a conintegrated time series regression problem described in page 11 of these notes by Eric Sims. There, taking first differences of two cointegrated random walks results in bias in the estimation stage, because the error in the difference vs difference regression includes a correction term to bring things in line with the long run relationship. If you rid yourself of that long term cointegrating relationship by differencing, you'll suffer bias.

Answer 2

This tries to answer the original question and not get into Marcos's paper etc. If you think that the level of a variable ( say log price ) has information, then differencing the series ( to obtain returns ), throws out information. If you don't think that the level has information, then differencing is fine. Engle and Granger in their 1987 econometric paper showed how it is possible to consider both levels and changes in the relationship between two series ( X and Y) through the use of an ECM. But it does not mean that there can't be cases where one doesn't care about the levels and is only interested in changes ( or vice versa ).

On a different note, here's a piece of advice. Whenever you read anything about strategies and techniques and approaches in finance don't put too much weight in them because, if the author REALLY TRULY HAS SOMETHING THAT WORKS, he-she is not going to divulge it anyway. Most of the stuff you read will be purposely vague and general and unless you know the details of what the person actually does, not terribly useful. That's not to say that Marcos doesn't write interesting papers but he's not going to tell you what he actually does so it's best to read his or anyone's presentations with that in mind.

Answer 3

The signal processing approach to such problems may be easier to understand. Among other things, the signal to noise ratio (SNR) for stocks may be so high as to render signal detection difficult, there are none-the-less many off-the-shelf algorithms for improving SNR, and that is perhaps of more interest to you. Such algorithms may sacrifice some signal (lossy algorithms) to accomplish noise reduction. The simplest algorithm to understand is perhaps addition. For example, if the trends are low frequency (long term) then reduction of the higher frequency noise by averaging (read as addition of) adjacent temporal signals or averaging over a short time 'window' will tend to reduce high frequency noise more so than low frequency signal thereby improving SNR. In the frequency domain this is called a low pass filter.

With respect to the question, addition and subtraction reduce information in the sense that when an average or difference is created, one loses the information as to what the numbers were before combining them. For example, given a $5$ as a result that could have come from $6-1$, $10-5$ or an uncountably infinite number of other number pairs.

Regarding information content such things are context dependent. If we assume that the noise contains no information, and that only the signal has information content, the goal of signal processing would then be to isolate the signal with the least possible signal loss while reducing the memoryless noise to zero. On the other hand, if the context is that both signal and noise contain information, then obviously noise reduction would seriously reduce information content.

With respect to prediction, if the noise contains no information then one could establish bounds or an 'envelope' in which the noise is contained. That is not exactly a prediction in the same sense as $y=f(t)$, but when coupled with the low pass signal may none-the-less offer a prediction range.

In the real world (assuming money is real) the high frequency data contains less information (but still some) and the low frequency data relatively more information. Still, signal processing may be useful. For example, examining the data in the frequency domain may reveal repetitive occurrences that are not obvious in the data itself, and such a process (Fourier transform) is lossless.

Answer 4

It's a well-know trade off between explanatory and predictive modeling.

We don't really do the difference nowadays but when you try to fit a model, the goal at the beginning is very important.

I can have the best predictive model of all time with non sense coefficient estimates or have a really good explanatory model that predicts poorly.

So basically when you do a differenciation of your series, you are more in a explanatory case than predictive so you sacrifice a little of the predictive power to have a better model in quality. Or at least you are ready to sacrifice but i don't think it can be proved that it will always make loose predictive power, maybe 95% of the cases base on personal experience.