Questions tagged [actor-critic]
The actor-critic tag has no usage guidance.
0
votes
0answers
19 views
Is it okay to calculate all the gradients for an LSTM at once?
I'm trying to use the AC method reported in "Asynchronous Methods for Deep Reinforcement Learning" for a project. The relevant algorithm is shown in pseudocode at the bottom, Algorithm S3.
I'm using ...
0
votes
0answers
38 views
Monte Carlo actor critic algorithm
Has anyone implemented Monte Carlo on policy actor-critic algorithm or know a codebase online and can share it?
P.S: I am not sure if this is the right place to post this, sorry for that.
2
votes
1answer
94 views
Understanding the temporal difference prediction error formula which uses a derivative
I'm very new to understanding the concept of prediction error underlying the output of the critic in the critic-actor method (RL learning), so bear with me, please.
For the temporal difference ...
3
votes
1answer
139 views
Reinforcement Learning - What is the logic behind actor-critic methods? Why use a critic?
Following David Silver's course, I came across the actor-critic policy improvement algorithm family.
It holds For one-step Markov decision processes that
$$\nabla_{\theta}J(\theta) = \mathbb{E}_{\...
1
vote
0answers
54 views
High variance of returns using Asynchronous Actor-Critic Agents (A3C) on CartPole [closed]
I ran the code from the Tensorflow blog with modified running average function (it takes a running mean of the last 3 episodes only) and notice strange behavior. Although the model achieves episode ...
1
vote
1answer
844 views
A2C Loss Function Explosion
I am training OpenAI's implementation of the A2C algorithm found here. Based on the mean episode reward graph below we can see it is in fact learning the policy function up until roughly 2000 updates:...
1
vote
1answer
440 views
What are the differences between contextual bandits, actor-citric methods, and continuous reinforcement learning?
Let's imagine we have a blackbox function f(X) -> y which we don't know. X is a vector of 10 continuous variables, which we ...
7
votes
1answer
2k views
Actor-critic loss function in reinforcement learning
In actor-critic learning for reinforcement learning, I understand you have an "actor" which is deciding the action to take, and a "critic" that then evaluates those actions, however, I'm confused on ...
