# Reinforcement learning based Q-learning for wireless routing

In the Q-learning method to get the optimal strategy, the update method is like the following: $$\begin{equation} Q(S,A) \leftarrow \ Q(S,A) + \alpha [R+\gamma~max_a(Q(s',a)) -Q(S,A)] \end{equation}$$

If in Q-learning the state transition probabilities are fixed to 1 then during the policy exploration how do we consider the $$\epsilon$$-greedy algorithm? As $$\epsilon$$ denotes the small probability during exploration and it must not be 1. Are the state transition probabilities and the probability during policy exploration different?

I would recommend reading Chapter 3 of Sutton and Barto's RL book. In it, you will learn about how reinforcement learning problems are formulated as a series of interactions between an agent and its environment. This is the key diagram. The "agent" is the entity that makes decisions. The "environment" is everything else, defined in the book as "everything outside of the agent's absolute control."

Notice that the environment responds to the agent's action with a reward $$R_{t+1}$$ and a next state $$S_{t+1}$$. Both of these are, in general, random variables with probability distributions conditioned on action $$A_t$$ and current state $$S_t$$.

In its attempt to behave optimally in a stochastic or uncertain environment, an agent has the option to introduce randomness of its own in the form of a stochastic policy. Here, a policy is a distribution over possible actions conditioned on the agent's current state. The agent samples from this distribution to decide what action to take.

Q-learning is a popular algorithm in RL, but I would suggest you start with reading the fundamentals (in the book I mentioned) before trying to implement any RL algorithms.

• Thank you for your comments. I am going through the given book. Mar 8 '19 at 6:55