0
$\begingroup$

Sutton et al. use throughout their book Reinforcement Learning capital letters to describe random variables. At page 131 they introduce Q-Learning.

$Q(S_t,A_t)\leftarrow Q(S_t,A_t) + \alpha [R_{t+1} + \gamma \max_a(S_{t+1},a) - Q(S_t,A_t)$

Why $S_t, A_t$ are capital letters and thus random variables? $S_t$ is the actual state where we are at the moment and $A_t$ is the action we executed. That said we have some concrete realisations $S_t=s$ and $A_t=a$.

$\endgroup$

1 Answer 1

0
$\begingroup$

Usually, in a Markov Decision Process, $S_t \text{ and } R_t$ are defined as random variables that depend only on the preceding state and action (Markov Property). $S_t \text{ and } R_t$ have well defined discrete probabilities, which determine the probability of assuming values at any given time step $S_t = s$ and $R_t = r$. Similarly, also the actions are defined as a random variable that depend on the current state $A(s)_t = a$.

$S_t$ is the actual state where we are at the moment and $A_t$ is the action we executed.

They define the probability of assuming a given value at time $t$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.