0
$\begingroup$

I am presented with the following Markov chain example:

A trader sells large and expensive machines.

$X_n$ is the number of machines in stock at the start of week $n$.

$D_n$ is the number of machines demanded by customers during week $n$.

Assume that $D_n \sim \text{Poi}(3)$, the $D_n$ are independent, and that $D_n$ and $X_n$ are independent for each $n$.

There are two stipulations:

  1. Inventory control: If $0$ or $1$ machines are left in stock by the end of a week, then machines are ordered and delivered to raise the stock to $5$ by the start of the next week. If $2$ or more machines are in stock at the end of the week, then no orders are placed.

  2. Lost business: If $D_n > X_n$, then the unsatisfied demands are lost.

We seek to show that $X_n$ is a Markov chain.

$(X_n - D_n)^+$ is the number of machines in stock at the end of week $n$.

$$X_{n + 1} \begin{cases} X_n - D_n & \text{if} X_n - D_n \ge 2 \\ 5 & \text{if} X_n - D_n \le 1 \end{cases}$$

$S = \{ 2, 3, 4, 5 \}$

Denote the history of the process up to time $n$ by $H_n = \{ X_0, X_1, \dots, X_n \}$.

Given that $X_n = i$, the independence assumptions ensure that $X_{n + 1}$ is conditionally independent of $H_{n - 1}$.

Case 1: $j = 2, 3,$ or $4$, and $i = j, \dots, 5$:

$$\begin{align} P(X_{n + 1} = j \vert X_n = i, H_{n - 1}) &= P(X_n - D_n = j \vert X_n = i, H_{n - 1}) \\ &= P(i - D_n = j \vert X_n = i, H_{n - 1}) \\ &= P(i - D_n = j) \\ &= P(D_n = i - j) \\ &= e^{-3} \dfrac{3^{i - j}}{(i - j)!} \end{align}$$

Case 2: $i = 2, 3,$ or $4$, and $j = 5$:

$$\begin{align} P(X_{n + 1} = 5 \vert X_n = i, H_{n - 1}) &= P(X_n - D_n \le 1 \vert X_n = i, H_{n - 1}) \\ &= P(i - D_n \le 1 \vert X_n = i, H_{n - 1}) \\ &= P(i - D_n \le 1) \\ &= P(D_n \ge i - 1) \\ &= 1 - P(D_n \le i - 2) \\ &=: p_{i5} \end{align}$$

Case 3: $i = 5, j = 5$:

$$p_{55} = P(D_n = 0) + P(D_n \ge 4) = P(D_n = 0) + 1 - P(D_n \le 3).$$

In case 2, the author has that $P(D_n \ge i - 1) = 1 - P(D_n \le i - 2) =: p_{i5}$. I don't understand how they concluded this. I would greatly appreciate it if people would please take the time to clarify this.

$\endgroup$
1
  • $\begingroup$ With which part? The first equality or the definition? $\endgroup$
    – Dale C
    Mar 12, 2020 at 22:50

1 Answer 1

0
$\begingroup$

If you are talking about the first equality:

$P(D_n \ge i - 1) = 1 - P(D_n \le i - 2)$

$D_n$ is Poisson distributed, so it's discrete. Therefore the probability of the RV being greater than or equal to some value is simply just $1$ minus it's CDF of the discrete value just before it (i.e. $i-2$).

Perhaps it is easier to see if you just move the $1$ to it's own side:

$P(D_n \ge i - 1) + P(D_n \le i - 2) = 1$

The sum of probabilities for all possible values is simply $1$.

If you are talking about the definition, well that is just how $p_{i5}$ is defined, from the first probability in the very first line.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.