1
$\begingroup$

An MDP (markov decision process) is defined as a set of states $S$, actions space $A$, Transition Probabilities $T$ and Rewards $R$. An action $a$ in a state $s$ usually result in a change of state and the probability is defined as $p(s'|s,a)$. I have a question if an action $a$ in a state $s$ does not result in a change of state then how to define transition probabilities $p(s'|s)$? How to define state transitions from $s$ to $s'$ when state transitions do not occur after taking an action $a$ in a state $s$?

$\endgroup$

1 Answer 1

1
$\begingroup$

You could always have a transition function which all the states transition back to themselves with probability 1:

$$P(s' = U | s = V, a) = \mathbf{1}(U = V)$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.