Mutual information between a time series and its delayed form

Question

According to Thomas Schreiber in the paper "Measuring Information Transfer", we need to calculate the Mutual Information between time series $t$ and itself with a delay of $k$ and let's call the delayed form $t^k$.

What I understand is that if $t=\{t_1, t_2, ..., t_n\}$, $t^k$ will be $\{t_{k+1},t_{k+2}, ..., t_{n}\}$. Therefore, their lengths are not the same. How can I compute the mutual information between these two time series? Do I have to pre-zero-pad $t^k$ or something like that?

Any help or reference in appreciated.

Answer 1

Using your notation, if you have a time series of length $n$ and you want to compute the mutual information between this time series and the time series delayed by $k$ positions, you should match $n-k$ values: $$t_1 , t_{k+1}\\ t_2 , t_{k+2}\\ \vdots \\ t_{n-k} , t_{n} $$ Mutual information between delayed time series is often computed to infer biological networks (See here and here).

So you will compute mutual information on $n-k$ data points rather than $n$. Be careful though when comparing mutual information estimated on a different number of data points: its estimates might be inflated if $n$ is small (See here).