Suppose $\{X_n : n =0,1,2,\dots \}$ is a DTMC that represents the inventory level at the end of day $n$. We have inventory policy $(2,4)$, i.e., if $X_n < 2$, we order enough units to have inventory level equal to $4$ by the beginning of the next day. We have the following demand curve: $P(D=1) = 1/6, P(D=2) = 3/6, P(D=3) = 2/6$. We assume demand is iid and demand is lost for an item if it is not in stock. Then the goal is calculate the following probability: $$P(X_1 = 3, X_4 = 2 | X_0 = 2)$$ I attempted to rewrite this as $P(X_1 = 3, X_4 = 2 | X_0 = 2) = P(X_4=2|X_1 = 3, X_0 = 2)P(X_1=3|X_0=2)$. So now I can calculate $P(X_1=3|X_0=2)$ using a transition matrix, but I am having trouble handling $P(X_4=2|X_1=3,X_0=3)$ since we do not know $X_3$ meaning we cannot apply the Markov property. Would something like law of total probability work here where I vary $X_3 =i$ across the sample space?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
You can apply implications of Markov property: $$P(X_4=2|X_1=3,X_0=3)=P(X_4=2|X_1=3)$$ And, then you can use the total probability law (as you also mentioned) for $X_3$ and then for $X_2$. But, this is already derived in theory, and is called as $3$-step probability here ($X_1$ to $X_4$), whose transition matrix is equal to $P^3$, and the relevant entry in this matrix (either $(1,4)$ or $(4,1)$ depending on your transition matrix definition, i.e. columnar or row-wise) will be the probability.
