# Calculating the transfer entropy in R

The transfer entropy, from information theory, is an effective way to measure the one-way information dependence between two variables. A nice high-level summary is here: http://lizier.me/joseph/presentations/20060503-Schreiber-MeasuringInfoTransfer.pdf

I see that there is a package for entropy and mutual information estimation (http://strimmerlab.org/software/entropy/), but not the one-way transfer metric.

What is an efficient way to calculate this in R? Perhaps I can use a chart output or metric from the mutual information package as a startpoint.

## migrated from stackoverflow.comJul 2 '11 at 4:57

This question came from our site for professional and enthusiast programmers.

• Learn to use google. typing transfer entropy + R leads to the following link on top http://users.utu.fi/attenka/TEpresentation081128.pdf which contains the code that you are seeking! – Ramnath Jun 30 '11 at 13:53

the same as above from the same page http://users.utu.fi/attenka/trent.R

###############################
###############################
## FUNCTION TRANSFER ENTROPY ##
###############################
###############################

# 070527 (ver. 081126), Atte Tenkanen
# s, time shift
trent<-function(Y,X,s=1){

#---------------------------------#
# Transition probability vectors: #
#---------------------------------#

L4=L1=length(X)-s # Lengths of vector Xn+1.
L3=L2=length(X) # Lengths of vector Xn (and Yn).

#-------------------#
# 1. p(Xn+s,Xn,Yn): #
#-------------------#

TPvector1=rep(0,L1) # Init.

for(i in 1:L1)
{
}

TPvector1T=table(TPvector1)/length(TPvector1) # Table of probabilities.

#-----------#
# 2. p(Xn): #
#-----------#

TPvector2=X
TPvector2T=table(X)/sum(table(X))

#--------------#
# 3. p(Xn,Yn): #
#--------------#

TPvector3=rep(0,L3)

for(i in 1:L3)
{
}

TPvector3T=table(TPvector3)/length(TPvector2)

#----------------#
# 4. p(Xn+s,Xn): #
#----------------#

TPvector4=rep(0,L4)

for(i in 1:L4)
{
}

TPvector4T=table(TPvector4)/length(TPvector4)

#--------------------------#
# Transfer entropy T(Y->X) #
#--------------------------#

SUMvector=rep(0,length(TPvector1T))
for(n in 1:length(TPvector1T))
{
SUMvector[n]=TPvector1T[n]*log10((TPvector1T[n]*TPvector2T[(unlist(strsplit(names(TPvector1T)[n],"i")))[2]])/(TPvector3T[paste((unlist(strsplit(names(TPvector1T)[n],"i")))[2],"i",(unlist(strsplit(names(TPvector1T)[n],"i")))[3],sep="",collapse="")]*TPvector4T[paste((unlist(strsplit(names(TPvector1T)[n],"i")))[1],"i",(unlist(strsplit(names(TPvector1T)[n],"i")))[2],sep="",collapse="")]))
}
return(sum(SUMvector))
} # End of the trent-function.


The JIDT toolkit which is the successor to the Matlab code in my high level summary linked in the original question, provides transfer entropy estimators for both discrete and continuous data, including various estimators for continuous data (Gaussian, box-kernel, Kraskov).

It can be used to calculate transfer entropy in R; this is carried out via the standard rJava package (R-to-Java interface).

The JIDT wiki pages describe how to get start using JIDT in R and provide several code examples.

See the .pdf found by Ramnath in comments section: http://users.utu.fi/attenka/TEpresentation081128.pdf

• for the sake of interwebs posterity, can we at least get some details on what is found in that PDF document so when that link is no longer stable, this answers doesn't become null and void? – Chase Aug 16 '11 at 2:24
• Good point. I uploaded the R-code snapshot (it is not in text form) that demonstrates how to calculate transfer entropy – Ram Ahluwalia Aug 16 '11 at 3:14

Would this:

https://cran.r-project.org/web/packages/TransferEntropy/TransferEntropy.pdf

help as well? The example makes sense sense but I am personally not sure how to measure significance.

• Unfortunately, the TransferEntropy package is archived as the authors did not provide fixes to some breaking issues. – David May 10 at 8:18

There is also the RTransferEntropy package, which allows the calculation of Shannon and Renyi TE measures and provides significance measures. The package uses C++ internally, so it should be reasonably fast.

Its also easy to use using the transfer_entropy(x, y) function for the x->y and y->x directions as well as significance levels. If you want to only get the x->y direction, you can use the shorter calc_te(x, y) function.

More examples can be found on the package's GitHub Page.

(Disclaimer, I am one of the authors of the package).