What is Combinatorial Purged Cross-Validation for time series data?

Question

I'm trying to understand the "Combinatorial Purged Cross-Validation" technique for time series data described in Marcos Lopez de Prado's "Advances in Financial Machine Learning" book (p. 163).

The setup is described as the researcher wanting to test "a number $\phi$ of backtest paths." I'm not really sure what that means, but here's what I have so far:

• A time series is split into $N$ sequential groups
• A number $k$ is chosen for cross validation
• A combinatoric equation is used to calculate the "number of paths": $$ \phi(N, k) = \frac{k}{N}{N \choose N - k}. $$

For the case of $N=6$ groups and $k=2$, there are $\phi(6, 2)=5$ paths and Figure 12.1 from the book lays them out as a table. The number of train / test CV split" is 15 (6 choose 2), which are indexed as the columns in the table below. The rows are the 6 groups, and the numbers inside are the path ids from 1 to 5.

The book states, "Path 2 is the result of combining forecasts from (G1,S2), (G2,S6), (G3,S6), (G4,S7), (G5,S8) and (G6,S9)." The passage of time through the G-groups, I can see. What I'm not following is how the splits relate to the groups.

People obviously think highly of this book. Here's a video of someone explaining Combinatorial Purged Cross Validation, but it didn't answer my questions. Can anybody tell me what's going on here? Is this truly an advancement over Walk Forward Cross Validation?

Answer 1

[Edit 2023-08-17: I answered this question myself after a while with no activity. Nice to see it's picked up, but I forgot a lot. I had to remind myself that this is all one time series, and that the time groups are on the vertical (1, 2, 3, 4). The horizontal (1,...,6) would be the CV index. I don't know what the intuitive explantion for the paths are, except I can see how they are combinatorially exhaustive].

Starting with a sketch illustrating the 3 paths in the $N = 4, k =2$ situation:

The number of ways to arrange the 2 test sets to occur in 4 time periods is ${4 \choose 2} = 6$, and $\frac{k}{N} = .5$ is the fraction of the combinations that will start with a test set. Since a "path" is a continuous group of blocks from the first to the last sequential group, there are .5 * 6 = 3 paths, which aligns with $\phi(N, k)$ from the question.

Here's a sketch for a more complicated example with N = 5 and k = 2, which leads to 4 paths:

Answer 2

I had the same question. I asked Marcos López de Prado, the person who created that methodology, on Twitter. Here is a link to his response.

Suppose that you have $6$ folds. A CV where you leave $2$ folds out (instead of the standard 1) at each split allows you to compute $5$ estimates for each datapoint. Instead of one PnL line, you can now estimate $5$ PnL lines.