I am unable to understand concepts related to the probability distribution of binary time series. This is from the book Binary time series by Benjamin Kedem, vol 52

Let $X_t$, t =0,1,... be a binary stationary Markov chain of $p^{th}$ order, such that $Pr(X_t = 1) = p, Pr(X_t=0) = q,$

$p+q =1$

We define, $Pr(X_t = x_t|X_{t-1} = x_{t-1},\ldots,X_{t-k} = x_{t-k}) = p_{x_t x_{t-1}\ldots x_{t-k}}$ Consider a stationary AR(1) process $Z_t = \phi Z_{t-1} + u_t$, where $|\phi| < 1$ and $u_t$ are independent $N(0,\sigma^2)$ variates. Then $Z_t$ is a zero mean stationary Gaussian process. Let $X_t$ be the clipped series at level zero. The joint distribution of the binary series of length (tuples) $n$ is given by $Pr(X_1 = x_1,\ldots,X_n = x_n) = \frac{1}{2} \prod_{i=2}^{n} \prod p_{y_iy_{i-1}\ldots y_1}^{I_i I_{i-1}\ldots I_1}$ Eq(1)

such that the second product is over all $2^i$ 0-1 tuples $(y_i,y_{i-1},\ldots,y_1)$ and $I_i = x_i$ if $y_i = 1$ otherwise $I_i = 1- x_i$ if $y_i = 0$

This is the joint distribution of any 0-1 time series $X_1,\ldots,X_n$ provided $Pr(X_1 = x_1) = 1/2$ and the products $I_tI_{t-1}\ldots I_1$ are the sufficient statistics for the chain.

My problem is that I have never seen any notation given in Eq(1) and I am unfamiliar with this kind of distribution where there is something on the power of probability. Can somebody please explain to me what is the meaning of the expression in Eq(1) and the notations $I$, power of probability? I remember reading about binomial distribution where the probability is raised to a power. But, this distribution and the meaning is complex to understand. Thank you for help.

  • 1
    $\begingroup$ There are several inconsistencies in the question. First it says that the Markov chain has order $p$. Then it uses $p$ as the probability of $X_t=1$. Then it gives a conditional distribution that contradicts what was just said, and has order $k$ instead of $p$. Note $p$ has $k+1$ subscripts here. Then Eq(1) uses $p$ with a varying number of subscripts, ranging from 2 to $n$. If these mistakes are all in the book, then I can understand your confusion. $\endgroup$
    – Tom Minka
    Nov 10, 2014 at 18:26

1 Answer 1


It looks like Eq(1) is just a (very confusing) way of writing the chain rule of probability: $$ Pr(X_1 = x_1, ..., X_n = x_n) = Pr(X_1 = x_1) Pr(X_2 = x_2 | X_1 = x_1) Pr(X_3 = x_3 | X_1 = x_1, X_2 = x_2) \cdots \\ = \frac{1}{2} \prod_{i=2}^n Pr(X_i = x_i | X_{i-1} = x_{i-1}, ..., X_1 = x_1) $$ This is because $$ \prod_{y_i, ..., y_1} p_{y_i \cdots y_1}^{I_i \cdots I_1} = p_{x_i \cdots x_1} $$ according to the definition of $I_i$. Consider the case when all $y$'s are one. Then each $I_i = x_i$, so $$ p_{y_i \cdots y_1}^{I_i \cdots I_1} = p_{1 \cdots 1}^{x_i \cdots x_1} $$ If any $x_j$ in the exponent is zero, the whole exponent is zero. Since any probability raised to the zeroth power equals 1, this factor would contribute nothing to the product. It only contributes when all $x_i, \cdots, x_1$ equal 1. Applying the same argument for any pattern of $y$ values shows that the only factor that contributes to the product is the one where $y_i = x_i, ..., y_1 = x_1$. This is how you get the equation above.

  • $\begingroup$ Thank you for your anser, however things are still unclear. Could you please clarify? (A) So there is no need for representing power of probability in Eq(1)?(B) Could you please elaborate why there are 2 products in Eq(1) and why there is only a single in your representation? (C)In your answer is there a typo $"<i"$ ? $\endgroup$
    – Ria George
    Nov 10, 2014 at 21:32
  • $\begingroup$ I've updated the answer to clarify these things. $\endgroup$
    – Tom Minka
    Nov 12, 2014 at 19:23
  • $\begingroup$ Thank you for the update and your effort. I still fail to understand what the $I's$ are and how come $\prod p_{yi..y1}^{I_1..I_1} = p_{xi..x1}$. This is because I don't understand the definition of $I$ and have fond no reference where I can clarify this concept. As you have followed, could you please share your knowledge in this limited space? Shall highly appreciate your time and effort. $\endgroup$
    – Ria George
    Nov 13, 2014 at 3:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.