I have a initial two state transition probability matrix, say A = [0.5 0.5; 0.7 0.3]. I want to create a Markov chain with 10 steps. In which I want that the chain should not have more than 40 % of state 1 and the remaining 60% to be in state 2.

For example: I want a Markov chain like this: 1 1 2 2 1 1. As state 1 has occurred 40% of the time so now it should not occur in the chain and the remaining chain should be like 2 2 2 2. That is, the final chain should, for this simulation, looks like 1 1 2 2 1 1 2 2 2 2.

Any solution even near to imposing the global limitations is also acceptable.

Thanks in Advance

  • 3
    $\begingroup$ It is unclear what you are attempting to model. Wouldn't a sequence of ten states with such a global limitation necessarily violate the Markov property? Could you clarify your probability model for the chain you are trying to create or simulate? $\endgroup$
    – whuber
    Jun 12, 2017 at 14:10

1 Answer 1


Let's label your initial states $1$ and $2$. You can introduce new set of states such as $(1,z)$ and $(2,z)$ where $z$ is the number of visits in state $1$. The transition probabilities change accordingly: $$ P(x_{t+1}=(1,z+1)|x_{t}=(1,z)) = 50\% $$ and $$ P(x_{t+1}=(2,z)|x_{t}=(1,z)) = 50\% $$ Similarly: $$ P(x_{t+1}=(1,z+1)|x_{t}=(2,z)) = 70\% $$ and $$ P(x_{t+1}=(2,z)|x_{t}=(2,z)) = 30\% $$ These transitions are defined for $z\leq4$. Then it turns into $$ P(x_{t+1}=(2,4)) = 100\% $$ All other transitions (not mentioned above) are zero.

The initial state is e.g. $(1,0)$.

  • $\begingroup$ This is a clever approach, but could you explain how you found these transition probabilities? It would seem they are not uniquely determined by the question. $\endgroup$
    – whuber
    Jun 12, 2017 at 14:12
  • $\begingroup$ Hi whuber. The transition probabilities defined in the question is an example. I have a large dataset that will give me the real transition probabilities with multiple states. I need to iterate over this procedure for 10000 steps. $\endgroup$ Jun 12, 2017 at 18:17
  • $\begingroup$ @Karel Thank you so much for this solution. I really appreciate that. $\endgroup$ Jun 12, 2017 at 18:27
  • $\begingroup$ @Karel. Will you please share a book, article, paper or any link, with more description on this solution. I need to explain this in a detailed and generic manner for my college assignment. I will really appreciate any help. $\endgroup$ Jun 13, 2017 at 16:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.