0
$\begingroup$

I'm working on a project that compares how predictable/unpredictable individuals' actions are in terms of how they transition between actions. We consider their actions to part of a first-order Markov chain, where $X$ is a random variable describing their present action and $Y$ describes their next action. We calculate $P(Y|X)$ based on observed data and then compute $H(Y|X)$ for each person.

Ideally, we'd like to be able to compare the transitional/behavioral unpredictability between individuals. Our concern is: we have $n$ human-defined "types" of actions, so some individuals engage in the full set of actions while some only perform a subset of them. Thus, the "maximum" possible conditional entropy differs between people: $|Y| \neq n$ for everyone. This makes me think that $H(Y|X)$ can't be compared between individuals as-is.

My first idea was to use Mutual Information instead because it accounts for how much uncertainty is contained in $Y$ in the first place: $I(X;Y)=H(Y)-H(Y|X)$. However, the maximum value of $H(Y)$ is also still affected the number of action categories. Instead, I'm thinking of proposing that we compare $\frac{H(Y|X)}{H(Y)}$ between participants: the percentage of uncertainty left in $Y$ after viewing $X$. Thus, we can compare how informative a person's present action is when predicting their subsequent action; less predictable people will have a greater percentage of the original entropy $H(Y)$ remaining than more predictable people.

However, I've been unable to find any mention/usage of the quantity $\frac{H(Y|X)}{H(Y)}$ in other papers. Is there a term for this value, or is there a reason why I can't find it anywhere? The closest comparable quantity I could find divides by maximum possible entropy instead ($\log(n)$), but this makes me uncomfortable because it requires us to assume that each person is only capable of the actions seen in the data and not all $n$ action categories.

$\endgroup$

1 Answer 1

0
$\begingroup$

EDIT: $\frac{H(Y|X)}{H(Y)}$ is just $1-U(Y|X)$ where $U$ is the uncertainty coefficient: https://en.wikipedia.org/wiki/Uncertainty_coefficient.

The actual formula is $$\frac{I(X;Y)}{H(Y)} = \frac{H(Y)-H(Y|X)}{H(Y)}$$ and describes the fraction of information/bits you can predict from seeing $Y$. It's a form of normalized mutual information.

I cannot believe how easy it was to find after typing my question up

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.