I am interested in finding the difference of entropy between two states of the same variable. The probability distribution associated with the variable X changes at each discrete point in time t by an increment in p(i). The current state is therefore always an update of the former state as we advance each step along the message. I am applying the equation below, where t is a multiplier to make the difference of entropy proportional to time (this is part of a study on statistical memory, so such proportionality gives a weight to events happening further along in a message).
My question is: what is I? I am inclined to call it information gain or information divergence, but the literature seems to be highly divergent (no pun intended) on the nomenclature. Of course I could be something different entirely (covariance?).
This is a question from someone with no formal background in statistics or information theory, so please bear with me while I try to make sense of my problem. Any clarification or suggestion will be highly appreciated.