Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
Sutton et al. use throughout their book Reinforcement Learning capital letters to describe random variables. At page 131 they introduce Q-Learning.
$Q(S_t,A_t)\leftarrow Q(S_t,A_t) + \alpha [R_{t+1} + \gamma \max_a(S_{t+1},a) - Q(S_t,A_t)$
Why $S_t, A_t$ are capital letters and thus random variables? $S_t$ is the actual state where we are at the moment and $A_t$ is the action we executed. That said we have some concrete realisations $S_t=s$ and $A_t=a$.
Usually, in a Markov Decision Process, $S_t \text{ and } R_t$ are defined as random variables that depend only on the preceding state and action (Markov Property). $S_t \text{ and } R_t$ have well defined discrete probabilities, which determine the probability of assuming values at any given time step $S_t = s$ and $R_t = r$. Similarly, also the actions are defined as a random variable that depend on the current state $A(s)_t = a$.
$S_t$ is the actual state where we are at the moment and $A_t$ is the action we executed.
They define the probability of assuming a given value at time $t$.
