# Limiting the total number of visits to a particular state in Markov chain

I have a initial two state transition probability matrix, say A = [0.5 0.5; 0.7 0.3]. I want to create a Markov chain with 10 steps. In which I want that the chain should not have more than 40 % of state 1 and the remaining 60% to be in state 2.

For example: I want a Markov chain like this: 1 1 2 2 1 1. As state 1 has occurred 40% of the time so now it should not occur in the chain and the remaining chain should be like 2 2 2 2. That is, the final chain should, for this simulation, looks like 1 1 2 2 1 1 2 2 2 2.

Any solution even near to imposing the global limitations is also acceptable.

Thanks in Advance

• It is unclear what you are attempting to model. Wouldn't a sequence of ten states with such a global limitation necessarily violate the Markov property? Could you clarify your probability model for the chain you are trying to create or simulate? – whuber Jun 12 '17 at 14:10

## 1 Answer

Let's label your initial states $1$ and $2$. You can introduce new set of states such as $(1,z)$ and $(2,z)$ where $z$ is the number of visits in state $1$. The transition probabilities change accordingly: $$P(x_{t+1}=(1,z+1)|x_{t}=(1,z)) = 50\%$$ and $$P(x_{t+1}=(2,z)|x_{t}=(1,z)) = 50\%$$ Similarly: $$P(x_{t+1}=(1,z+1)|x_{t}=(2,z)) = 70\%$$ and $$P(x_{t+1}=(2,z)|x_{t}=(2,z)) = 30\%$$ These transitions are defined for $z\leq4$. Then it turns into $$P(x_{t+1}=(2,4)) = 100\%$$ All other transitions (not mentioned above) are zero.

The initial state is e.g. $(1,0)$.

• This is a clever approach, but could you explain how you found these transition probabilities? It would seem they are not uniquely determined by the question. – whuber Jun 12 '17 at 14:12
• Hi whuber. The transition probabilities defined in the question is an example. I have a large dataset that will give me the real transition probabilities with multiple states. I need to iterate over this procedure for 10000 steps. – Amit Kumar Jain Jun 12 '17 at 18:17
• @Karel Thank you so much for this solution. I really appreciate that. – Amit Kumar Jain Jun 12 '17 at 18:27
• @Karel. Will you please share a book, article, paper or any link, with more description on this solution. I need to explain this in a detailed and generic manner for my college assignment. I will really appreciate any help. – Amit Kumar Jain Jun 13 '17 at 16:11