1
$\begingroup$

Transfer entropy (TE) measures the amount of directed transfer of information between two random processes, $X$ and $Y$. In other words, through the TE we can quantify the amount of information that flows from $X$ to $Y$ and viceversa.

However, it's my understanding that TE doesn't tell us whether the relationship is positive or negative, i.e. if an increase (decrease) in $X$ decreases (increases) $Y$. Is this the case? How could we estimate the sign of relationship between $X$ and $Y$ by using TE?

$\endgroup$
1
$\begingroup$

Transfer entropy $TE(X\mapsto Y)$ only informs you about how the history of $X$ has an influence on the current state of $Y$. Transfer entropy, by definition, can never be negative and as you suspected, such semantic account of the relation between two random variables cannot be given by transfer entropy.

However, there is another concept that you might find useful, called as the local transfer entropy. It is basically defined as the pointwise conditional mutual information,

$$te_{X\mapsto Y}(t)=i(y_t;x_{t-1}|y_{t-1}) = log_2\frac{p(y_t|x_{t-1}, y_{t-1})}{p(y_t|y_{t-1})}.$$

If, say, the probability of having the next state of a destination given the value of its source is lower than the probability of having that state independent of the source value, then the local transfer entropy is negative. You can interpret that as having some sort of "deceptive" information - this then could be seen as a relation of negative nature for a specific event, as you (I believe) tried to exemplify in your question.

For more on these, you might like to check this book dedicated to transfer entropy.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.