# Conditional probabilities with Discrete Time Markov Chain non-perishable inventory

Suppose $$\{X_n : n =0,1,2,\dots \}$$ is a DTMC that represents the inventory level at the end of day $$n$$. We have inventory policy $$(2,4)$$, i.e., if $$X_n < 2$$, we order enough units to have inventory level equal to $$4$$ by the beginning of the next day. We have the following demand curve: $$P(D=1) = 1/6, P(D=2) = 3/6, P(D=3) = 2/6$$. We assume demand is iid and demand is lost for an item if it is not in stock. Then the goal is calculate the following probability: $$P(X_1 = 3, X_4 = 2 | X_0 = 2)$$ I attempted to rewrite this as $$P(X_1 = 3, X_4 = 2 | X_0 = 2) = P(X_4=2|X_1 = 3, X_0 = 2)P(X_1=3|X_0=2)$$. So now I can calculate $$P(X_1=3|X_0=2)$$ using a transition matrix, but I am having trouble handling $$P(X_4=2|X_1=3,X_0=3)$$ since we do not know $$X_3$$ meaning we cannot apply the Markov property. Would something like law of total probability work here where I vary $$X_3 =i$$ across the sample space?

You can apply implications of Markov property: $$P(X_4=2|X_1=3,X_0=3)=P(X_4=2|X_1=3)$$ And, then you can use the total probability law (as you also mentioned) for $$X_3$$ and then for $$X_2$$. But, this is already derived in theory, and is called as $$3$$-step probability here ($$X_1$$ to $$X_4$$), whose transition matrix is equal to $$P^3$$, and the relevant entry in this matrix (either $$(1,4)$$ or $$(4,1)$$ depending on your transition matrix definition, i.e. columnar or row-wise) will be the probability.