3
$\begingroup$

Suppose $\{X_n : n =0,1,2,\dots \}$ is a DTMC that represents the inventory level at the end of day $n$. We have inventory policy $(2,4)$, i.e., if $X_n < 2$, we order enough units to have inventory level equal to $4$ by the beginning of the next day. We have the following demand curve: $P(D=1) = 1/6, P(D=2) = 3/6, P(D=3) = 2/6$. We assume demand is iid and demand is lost for an item if it is not in stock. Then the goal is calculate the following probability: $$P(X_1 = 3, X_4 = 2 | X_0 = 2)$$ I attempted to rewrite this as $P(X_1 = 3, X_4 = 2 | X_0 = 2) = P(X_4=2|X_1 = 3, X_0 = 2)P(X_1=3|X_0=2)$. So now I can calculate $P(X_1=3|X_0=2)$ using a transition matrix, but I am having trouble handling $P(X_4=2|X_1=3,X_0=3)$ since we do not know $X_3$ meaning we cannot apply the Markov property. Would something like law of total probability work here where I vary $X_3 =i$ across the sample space?

$\endgroup$
1
$\begingroup$

You can apply implications of Markov property: $$P(X_4=2|X_1=3,X_0=3)=P(X_4=2|X_1=3)$$ And, then you can use the total probability law (as you also mentioned) for $X_3$ and then for $X_2$. But, this is already derived in theory, and is called as $3$-step probability here ($X_1$ to $X_4$), whose transition matrix is equal to $P^3$, and the relevant entry in this matrix (either $(1,4)$ or $(4,1)$ depending on your transition matrix definition, i.e. columnar or row-wise) will be the probability.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.