# Case 2 of Markov chain example: $P(D_n \ge i - 1) = 1 - P(D_n \le i - 2) =: p_{i5}$

I am presented with the following Markov chain example:

A trader sells large and expensive machines.

$$X_n$$ is the number of machines in stock at the start of week $$n$$.

$$D_n$$ is the number of machines demanded by customers during week $$n$$.

Assume that $$D_n \sim \text{Poi}(3)$$, the $$D_n$$ are independent, and that $$D_n$$ and $$X_n$$ are independent for each $$n$$.

There are two stipulations:

1. Inventory control: If $$0$$ or $$1$$ machines are left in stock by the end of a week, then machines are ordered and delivered to raise the stock to $$5$$ by the start of the next week. If $$2$$ or more machines are in stock at the end of the week, then no orders are placed.

2. Lost business: If $$D_n > X_n$$, then the unsatisfied demands are lost.

We seek to show that $$X_n$$ is a Markov chain.

$$(X_n - D_n)^+$$ is the number of machines in stock at the end of week $$n$$.

$$X_{n + 1} \begin{cases} X_n - D_n & \text{if} X_n - D_n \ge 2 \\ 5 & \text{if} X_n - D_n \le 1 \end{cases}$$

$$S = \{ 2, 3, 4, 5 \}$$

Denote the history of the process up to time $$n$$ by $$H_n = \{ X_0, X_1, \dots, X_n \}$$.

Given that $$X_n = i$$, the independence assumptions ensure that $$X_{n + 1}$$ is conditionally independent of $$H_{n - 1}$$.

Case 1: $$j = 2, 3,$$ or $$4$$, and $$i = j, \dots, 5$$:

\begin{align} P(X_{n + 1} = j \vert X_n = i, H_{n - 1}) &= P(X_n - D_n = j \vert X_n = i, H_{n - 1}) \\ &= P(i - D_n = j \vert X_n = i, H_{n - 1}) \\ &= P(i - D_n = j) \\ &= P(D_n = i - j) \\ &= e^{-3} \dfrac{3^{i - j}}{(i - j)!} \end{align}

Case 2: $$i = 2, 3,$$ or $$4$$, and $$j = 5$$:

\begin{align} P(X_{n + 1} = 5 \vert X_n = i, H_{n - 1}) &= P(X_n - D_n \le 1 \vert X_n = i, H_{n - 1}) \\ &= P(i - D_n \le 1 \vert X_n = i, H_{n - 1}) \\ &= P(i - D_n \le 1) \\ &= P(D_n \ge i - 1) \\ &= 1 - P(D_n \le i - 2) \\ &=: p_{i5} \end{align}

Case 3: $$i = 5, j = 5$$:

$$p_{55} = P(D_n = 0) + P(D_n \ge 4) = P(D_n = 0) + 1 - P(D_n \le 3).$$

In case 2, the author has that $$P(D_n \ge i - 1) = 1 - P(D_n \le i - 2) =: p_{i5}$$. I don't understand how they concluded this. I would greatly appreciate it if people would please take the time to clarify this.

• With which part? The first equality or the definition? Mar 12, 2020 at 22:50

If you are talking about the first equality:

$$P(D_n \ge i - 1) = 1 - P(D_n \le i - 2)$$

$$D_n$$ is Poisson distributed, so it's discrete. Therefore the probability of the RV being greater than or equal to some value is simply just $$1$$ minus it's CDF of the discrete value just before it (i.e. $$i-2$$).

Perhaps it is easier to see if you just move the $$1$$ to it's own side:

$$P(D_n \ge i - 1) + P(D_n \le i - 2) = 1$$

The sum of probabilities for all possible values is simply $$1$$.

If you are talking about the definition, well that is just how $$p_{i5}$$ is defined, from the first probability in the very first line.