Canonical Time Warping is a state-of-the-art technique for time alignment. According to the original paper, it helps account for individual varieties when aligning sequences derived from different subjects performing similar event (e.g., aligning sensor data referred to two subjects' walking), by maximizing sequences correlation in a projected subspace.
Consider the iterative procedure of CTW that combines Canonical Coef. Analysis (CCA) and Dynamic Time Warping (DTW): Given a pair of temporally misaligned time-series that are in the same dimension, DTW is performed to initialize the warping matrices.
- Then, CCA is performed on the warped time-series (not necessarily well aligned) to give a pair of projection matrices.
- Multiply the original input (unwarped sequences) by the corresponding projection matrix obtained in 1, on which DTW is then performed. The warping matrices will be used in the next iteration.
- Before next iteration, warp (obtained in 2) the input and multiply it with projection matrices (obtained in 1), and assess the Euclidean distance of two sequences. The iteration stops as the Euclidean loss reaches a threshold.
My question is, how does CCA help time-alignmnet in first step of iteration? Since warping matrices given by DTW is not well tuned, it might not make sense to force maximal correlation between two not-well-aligned or misaligned sequences.
For example, consider an extreme case: two same, but out-of-phase time-sequences are firstly analyzed by CCA, followed by DTW. The resulting warping path can be diagonal (45 degree) if the projected sequences are flattened to zeros by CCA, which is totally wrong. Even if not zeros, making similar the two temporally misaligned time-series can harm the performance of DTW, so I see no benefit of CCA in CTW.
Could some give a straight forward explanation about it? Or correct my understanding of CTW, please.