# Q-Learning [Sutton]: why random variable in formula

Sutton et al. use throughout their book Reinforcement Learning capital letters to describe random variables. At page 131 they introduce Q-Learning.

$$Q(S_t,A_t)\leftarrow Q(S_t,A_t) + \alpha [R_{t+1} + \gamma \max_a(S_{t+1},a) - Q(S_t,A_t)$$

Why $$S_t, A_t$$ are capital letters and thus random variables? $$S_t$$ is the actual state where we are at the moment and $$A_t$$ is the action we executed. That said we have some concrete realisations $$S_t=s$$ and $$A_t=a$$.

Usually, in a Markov Decision Process, $$S_t \text{ and } R_t$$ are defined as random variables that depend only on the preceding state and action (Markov Property). $$S_t \text{ and } R_t$$ have well defined discrete probabilities, which determine the probability of assuming values at any given time step $$S_t = s$$ and $$R_t = r$$. Similarly, also the actions are defined as a random variable that depend on the current state $$A(s)_t = a$$.
$$S_t$$ is the actual state where we are at the moment and $$A_t$$ is the action we executed.
They define the probability of assuming a given value at time $$t$$.