Notation for conditional probability when the conditioned event is observed

I am trying to understand proper notation for functions related to state transition models (e.g., a HMM). There are two indexing variables $$t$$ (time step) and $$i$$ (state). Such that it matters, in my scenario, the variable we are conditioning on may be taken to be a realized (observed) instance of a random variable: $$P(X=x|Y=y)$$, not $$P(X=x|Y)$$. Below is an example, an entropy function over the transition model, but the equation could be anything including a sum over states and probability conditioned on previously observed states or similar:

$$H_t = -\sum P(x_{t}|x_{t-1})log_{2}P(x_{t}|x_{t-1})$$

I would call this correct but incomplete/informal, right? Importantly, it lacks the index variable for summation. I get a little confused when I attempt to add this in. Should it look like this:

$$H_t = -\sum_{i=1}^{n}P(x_{t,i}|x_{t-1})log_{2}P(x_{t,i}|x_{t-1})$$

This feels off to me in that both $$x_{t}$$ and $$x_{t-1}$$ have $$n$$ states. The index $$i$$ affixed to $$x_{t}$$ (i.e., $$x_{t,i}$$) provides a specificity greater than that for $$x_{t-1}$$, which lacks a state value designation. Given that $$x_{t-1}$$ is a realized (random) variable and $$x_{t}$$ is an (unrealized) random variable, shouldn't there be some notation to indicate the value of $$i$$ is realized (even more specific) with respect to $$x_{t-1}$$? If so, how? Surely not $$x_{t-1,1}$$ or $$x_{t-1,j}$$. Perhaps this is where the $$X=x$$ grammar is useful but, that seems only to displace the problem, ex.:

$$H_t = -\sum_{i=1}^{n}P(X_{t}=x_{i}|X_{t-1}=x)log_{2}P(X_{t}=x_{i}|X_{t-1}=x)$$

Clearly there is something I am missing. Please send thoughts. Math notation is not my strength, where I have seemingly or obviously stated something incorrectly or oddly, please let me know in the comments and I will respond and make edits.

FYI, here is somewhat related to this question.

$$P(x_t|x_{t-1})$$ is an abuse of notation to make it shorter. The unambiguous version is as you have noted: $$P(X_t=x_t|X_{t-1}=x_{t-1})$$. In graphical models, such as HMM, $$x_t$$ generally denotes the latent variable, i.e. the state, at time $$t$$. So, if there are $$n$$ states, $$x_t$$ can be one of $$1,2,...,n$$. Index $$i$$ can be used to denote one of them, e.g. $$x_t=i$$ means we're in state $$i$$ at time $$t$$.

It seems like $$H_t$$ is the conditional entropy of $$X_t$$ given a specific value of $$X_{t-1}$$, i.e. $$H(X_t|X_{t-1}=x_{t-1})$$. Then, the summation converts into the unambiguous form: $$H_t=\sum_{i=1}^n P(X_t=i|X_{t-1}=x_{t-1})\log_2P(X_t=i|X_{t-1}=x_{t-1})$$

Note that $$H_t$$ will be a function of $$x_{t-1}$$.

• Very clear. Thank you!
– jbjo
Commented Aug 4, 2019 at 19:54
• I like this answer but can we change the summation index to $x_t$: $$H_{t}=\sum_{x_{t}=1}^{n}P(X_{t}=x_{t}|X_{t−1}=x_{t−1})log_{2}P(X_{t}=x_{t}|X_{t−1}=x_{t−1})$$ In the answer, $j$ is an index value, taken from 1 to $n$ on the $n$ states of $X_t$. $P(X_t=j)$ implies that the RHS value in this grammar is an address value, not the value taken by $X_t$. I had thought common usage was the opposite, eg, $P(X=3)$ is the probability $X$ takes the value 3. Am I thinking about this incorrectly? Is "3" both a value and an address on the number line? Can you clarify this for me?
– jbjo
Commented Aug 4, 2019 at 23:52
• correction, in the answer, the index variable is $i$, not $j$...
– jbjo
Commented Aug 5, 2019 at 2:26