Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [actor-critic]

The tag has no usage guidance.

0
votes
0answers
19 views

Is it okay to calculate all the gradients for an LSTM at once?

I'm trying to use the AC method reported in "Asynchronous Methods for Deep Reinforcement Learning" for a project. The relevant algorithm is shown in pseudocode at the bottom, Algorithm S3. I'm using ...
0
votes
0answers
38 views

Monte Carlo actor critic algorithm

Has anyone implemented Monte Carlo on policy actor-critic algorithm or know a codebase online and can share it? P.S: I am not sure if this is the right place to post this, sorry for that.
2
votes
1answer
94 views

Understanding the temporal difference prediction error formula which uses a derivative

I'm very new to understanding the concept of prediction error underlying the output of the critic in the critic-actor method (RL learning), so bear with me, please. For the temporal difference ...
3
votes
1answer
139 views

Reinforcement Learning - What is the logic behind actor-critic methods? Why use a critic?

Following David Silver's course, I came across the actor-critic policy improvement algorithm family. It holds For one-step Markov decision processes that $$\nabla_{\theta}J(\theta) = \mathbb{E}_{\...
1
vote
0answers
54 views

High variance of returns using Asynchronous Actor-Critic Agents (A3C) on CartPole [closed]

I ran the code from the Tensorflow blog with modified running average function (it takes a running mean of the last 3 episodes only) and notice strange behavior. Although the model achieves episode ...
1
vote
1answer
844 views

A2C Loss Function Explosion

I am training OpenAI's implementation of the A2C algorithm found here. Based on the mean episode reward graph below we can see it is in fact learning the policy function up until roughly 2000 updates:...
1
vote
1answer
440 views

What are the differences between contextual bandits, actor-citric methods, and continuous reinforcement learning?

Let's imagine we have a blackbox function f(X) -> y which we don't know. X is a vector of 10 continuous variables, which we ...
7
votes
1answer
2k views

Actor-critic loss function in reinforcement learning

In actor-critic learning for reinforcement learning, I understand you have an "actor" which is deciding the action to take, and a "critic" that then evaluates those actions, however, I'm confused on ...