Consider a Markov decision process in which we transition from state $s_t \rightarrow s_{t+1}$ by taking action $a_t$, and then apply an update to a single entry from a table of $Q$-values based on a stochastic reward $r(s_t, a_t)$:
$$Q_{t+1}(s_t, a_t) = (1 - \alpha_t(s_t, a_t))Q_t(s_t, a_t) + \alpha_t(s_t, a_t)\left[ r(s_t, a_t) + \gamma Q_t(s_{t+1}, a_{t+1}) \right]$$
This is the update rule for $\texttt{SARSA}(0)$, which is an implementation of on-policy $Q$-learning. (*)
Can we assume the current reward is independent from the next state, or the current $Q$-values for the next state? In other words,
\begin{align} r(s_t, a_t) &\,\, \overset{?}{\perp\!\!\!\perp} \,\, s_{t+1} \\ r(s_t, a_t) &\,\, \overset{?}{\perp\!\!\!\perp} \,\, Q_t(s_{t+1}, \cdot) \end{align}
If not, what additional constraints are needed in order to make the weaker claim that they are uncorrelated? In the end, what I basically need to have is uncorrelatedness: $$E\left[ r(s_t, a_t) Q_t(s_{t+1}, \cdot) \right] = E\left[ r(s_t, a_t) \right]E\left[Q_t(s_{t+1}, \cdot)\right]$$
More generally, how can we determine this for various terms in such a stochastic process?
(*) Source: Convergence Results for Single-Step On-Policy Reinforcement-Learning Algorithms