7
$\begingroup$

The transfer entropy, from information theory, is an effective way to measure the one-way information dependence between two variables. A nice high-level summary is here: http://lizier.me/joseph/presentations/20060503-Schreiber-MeasuringInfoTransfer.pdf

I see that there is a package for entropy and mutual information estimation (http://strimmerlab.org/software/entropy/), but not the one-way transfer metric.

What is an efficient way to calculate this in R? Perhaps I can use a chart output or metric from the mutual information package as a startpoint.

Improve this question
$\endgroup$
0

5 Answers 5

Reset to default
1
$\begingroup$

the same as above from the same page http://users.utu.fi/attenka/trent.R

###############################
###############################
## FUNCTION TRANSFER ENTROPY ##
###############################
###############################

# 070527 (ver. 081126), Atte Tenkanen
# s, time shift
trent<-function(Y,X,s=1){

    #---------------------------------#
    # Transition probability vectors: #
    #---------------------------------#

    L4=L1=length(X)-s # Lengths of vector Xn+1.
    L3=L2=length(X) # Lengths of vector Xn (and Yn).

    #-------------------#
    # 1. p(Xn+s,Xn,Yn): #
    #-------------------#

    TPvector1=rep(0,L1) # Init.

    for(i in 1:L1)
    {
            TPvector1[i]=paste(c(X[i+s],"i",X[i],"i",Y[i]),collapse="") # "addresses"
    }

    TPvector1T=table(TPvector1)/length(TPvector1) # Table of probabilities.

    #-----------#
    # 2. p(Xn): #
    #-----------#

    TPvector2=X
    TPvector2T=table(X)/sum(table(X))

    #--------------#
    # 3. p(Xn,Yn): #
    #--------------#

    TPvector3=rep(0,L3)

    for(i in 1:L3)
    {
            TPvector3[i]=paste(c(X[i],"i",Y[i]),collapse="") # addresses
    }

    TPvector3T=table(TPvector3)/length(TPvector2)

    #----------------#
    # 4. p(Xn+s,Xn): #
    #----------------#

    TPvector4=rep(0,L4)

    for(i in 1:L4)
    {
            TPvector4[i]=paste(c(X[i+s],"i",X[i]),collapse="") # addresses
    }

    TPvector4T=table(TPvector4)/length(TPvector4)

    #--------------------------#
    # Transfer entropy T(Y->X) #
    #--------------------------#

    SUMvector=rep(0,length(TPvector1T))
    for(n in 1:length(TPvector1T))
    {
        SUMvector[n]=TPvector1T[n]*log10((TPvector1T[n]*TPvector2T[(unlist(strsplit(names(TPvector1T)[n],"i")))[2]])/(TPvector3T[paste((unlist(strsplit(names(TPvector1T)[n],"i")))[2],"i",(unlist(strsplit(names(TPvector1T)[n],"i")))[3],sep="",collapse="")]*TPvector4T[paste((unlist(strsplit(names(TPvector1T)[n],"i")))[1],"i",(unlist(strsplit(names(TPvector1T)[n],"i")))[2],sep="",collapse="")]))
    }
    return(sum(SUMvector))
} # End of the trent-function.
Improve this answer
$\endgroup$
1
$\begingroup$

The JIDT toolkit which is the successor to the Matlab code in my high level summary linked in the original question, provides transfer entropy estimators for both discrete and continuous data, including various estimators for continuous data (Gaussian, box-kernel, Kraskov).

It can be used to calculate transfer entropy in R; this is carried out via the standard rJava package (R-to-Java interface).

The JIDT wiki pages describe how to get start using JIDT in R and provide several code examples.

Improve this answer
$\endgroup$
1
$\begingroup$

There is also the RTransferEntropy package, which allows the calculation of Shannon and Renyi TE measures and provides significance measures. The package uses C++ internally, so it should be reasonably fast.

Its also easy to use using the transfer_entropy(x, y) function for the x->y and y->x directions as well as significance levels. If you want to only get the x->y direction, you can use the shorter calc_te(x, y) function.

More examples can be found on the package's GitHub Page.

(Disclaimer, I am one of the authors of the package).

Improve this answer
$\endgroup$
1
0
$\begingroup$

See the .pdf found by Ramnath in comments section: http://users.utu.fi/attenka/TEpresentation081128.pdf

enter image description here

Improve this answer
$\endgroup$
2
  • $\begingroup$ for the sake of interwebs posterity, can we at least get some details on what is found in that PDF document so when that link is no longer stable, this answers doesn't become null and void? $\endgroup$
    – Chase
    Commented Aug 16, 2011 at 2:24
  • $\begingroup$ Good point. I uploaded the R-code snapshot (it is not in text form) that demonstrates how to calculate transfer entropy $\endgroup$
    – Ram Ahluwalia
    Commented Aug 16, 2011 at 3:14
-1
$\begingroup$

Would this:

https://cran.r-project.org/web/packages/TransferEntropy/TransferEntropy.pdf

help as well? The example makes sense sense but I am personally not sure how to measure significance.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Unfortunately, the TransferEntropy package is archived as the authors did not provide fixes to some breaking issues. $\endgroup$
    – David
    Commented May 10, 2019 at 8:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.