# Questions tagged [q-learning]

A popular reinforcement learning algorithm, an instance of TD (temporal difference) learning.

163 questions
Filter by
Sorted by
Tagged with
22 views

### Double q learning

Can we expect that the two q tables converge together? which means that abs(Q1-Q2).max() converge to zero, Can we say that?
1 vote
33 views

### Convergence of the SARSA algorithm

I'm trying to figure out the convergence of the SARSA algorithm, but I need help. In the article "On the Convergence of Stochastic Iterative Dynamic Programming" by Jakkola, Jordana and ...
• 11
22 views

### Correct equation for Q learning

I start learning Q learning algorithm. However, I can not understand which is the correct equation for q learning. I found different equation from different sources. Source 1 Reinforcement Learning ...
• 175
106 views

### Several Questions About "Prioritized Experience Replay"

I have 3 questions about "Prioritized Experience Buffer" as described in the paper. what's the point of the importance sampling (IS)? I'll explain - I understand that: a. when we ...
• 243
1 vote
673 views

### What is the difference between an Epoch and an Episode?

I am working on DQN and have confused myself with Epoch and Episode. I have gone through this answer but the confusion is increased. I will explain my scenario and would like to understand the ...
• 113
1 vote
137 views

1 vote
338 views

### Expected SARSA, SARSA and Q-learning

I would much appreciate if you could point me in the right direction regarding this question about targets for approximate ...
• 111
1 vote
2k views

### Deep Q Learning best practice

I'm new in deep q-learning and I have understood its main concepts and I'm trying to solve problems with DQL. The problem is that I don't know how to initialize some key values (AKA hyperparameters) ...
• 119
1 vote
41 views

### Small difference of q-function between different actions for the same state

I am trying out reinforcement learning using Q-learning. The data come from some made-up equations so I have infinite number of data. One thing that troubles me is after I learn the Q-function, I use ...
• 2,013
815 views

### DQN - agent doesn't improve policy

I have a simple grid environment. The player is in the upper left corner and it's goal is to get to lower right corner. The player receives +0.2 points for moving in the direction of the goal, -0.2 ...
• 131
1k views

### Delayed Rewards in Reinforcement Learning

I have an MDP where the rewards are delayed for six steps as follows: The reward from action at time t is received when the action at time t+6 is taken. The reward from action at time t+1 is ...
• 741
53 views

### How to use data from other policies in order to find optimal policy in model-free rl?

I am struggling to understand whether experience from one policy can be used to find optimal policy. Suppose that I have gathered many data (state, action, reward, next_state) by following random ...
• 741
1 vote
2k views

### Q-learning agent stucks in an infinite loop

I am simulating a mouse to find a cheese on an empty table. I randomly put a cheese on the table and let the mouse find the cheese without falling off the table. The problem is, in test part, agent ...
• 445
83 views

### Rationale behind Q-learning

I am reading Sutton Barto on Reinforcement Learning. I understand that $TD(\lambda)$ methods propose better performance than Monte Carlo methods, with TD methods combining advantages of Dynamic ...
• 643
806 views

### Reward attribution in deep q learning and texas holdem poker

I’m having issues with reward attribution in poker using deep q learning. Multiple actions will yield one reward, but the reward is only known at the end of the hand, not before. I have built a gym ...
• 321
138 views

### Why not sample action from Q values?

When collecting experience from which to estimate a Q(s,a) function, one common technique in the literature is to follow an epsilon greedy-strategy. In this strategy, the agent selects a random action ...
• 155
1 vote