Newest 'actor-critic' Questions

1

vote

1answer

14 views

Deep deterministic policy gradient : which network do I have to use for testing?

We know that Deep deterministic policy gradient (henceforth ddpg) is characterized by two kind of neural networks: one related to the critic $Q$ the other to the actor $\mu$ with parameters $\theta^\...

asked Sep 10 at 20:56

Siderius

113 bronze badges

1

vote

0answers

14 views

Understanding the Neural Network Output using a Custom Gym Env continuous boundaries

I need help understanding some concepts on neural networks while using a Custom Gym Environment. ...

neural-networks reinforcement-learning actor-critic

asked Jun 19 at 2:28

uBoscuBo

113 bronze badges

0

votes

0answers

23 views

Why is my actor loss reversed?

I'm trying to train RL model with REINFORCE and a Critic network. The agent is learning, but I don't understand the look of my actor loss. Here are the curves: In this task the goal is to minimize ...

reinforcement-learning actor-critic

asked Mar 13 at 3:16

Astariul

1012 bronze badges

2

votes

1answer

181 views

How to define number of states in reinforcement learning

I'm a robotic engineer who's relatively new to reinforcement learning and I want to try to do simple reinforcement learning on a robot to optimize its velocity. I am however having trouble with ...

neural-networks markov-process reinforcement-learning temporal-difference actor-critic

asked Jun 15 '20 at 10:22

Mr_Melon

211 bronze badge

2

votes

3answers

42 views

Why the approximation of $\log \pi_{\theta}(a|s)$ improves numerical stability?

In Maxim Lapan's book Deep Reinforcement Learning Hands-on, section Continuous A2C, it says By definition, the probability density function of the Gaussian Distribution is $$f(x | \mu, \sigma^2) =...

reinforcement-learning policy-gradient actor-critic

asked May 24 '20 at 22:44

jgauth

413 bronze badges

2

votes

0answers

86 views

Can't use replay memory with policy gradient, why?

One of the approaches to improving the stability of the Policy Gradient family of methods is to use multiple environments in parallel. The reason behind this is the fundamental problem we ...

policy-gradient actor-critic

asked May 13 '20 at 13:26

jgauth

413 bronze badges

3

votes

0answers

347 views

A2C in TensorFlow 2 using model with two heads

I am implementing some of the basic reinforcement learning algorithms but ran into a problem with an online (one-step TD) A2C implementation where my reward seems to decrease over time instead of ...

reinforcement-learning tensorflow actor-critic

asked Jan 30 '20 at 0:09

Gregor

1536 bronze badges

3

votes

1answer

185 views

Advantage term in A2C

I am trying to understand A2C algorithm by an article Understanding Actor Critic Methods and A2C by Chris Yoon An author provides the following formula to compute an Advantage term: \begin{equation} ...

reinforcement-learning actor-critic

asked Oct 25 '19 at 4:19

koryakinp

1993 silver badges10 bronze badges

1

vote

0answers

224 views

entropy coefficient in A3C [closed]

I see that my policy entropy is decreasing very fast and converges to zero in no time, causing the policy to sample the same action again and again (which results in a sub-optimal behavior). I think a ...

reinforcement-learning loss-functions entropy actor-critic

asked Aug 20 '19 at 20:24

Gabizon

3781 silver badge11 bronze badges

2

votes

0answers

516 views

Reinforcement Learning: A2C agent does not learn

I am trying to implement an A2C algorithm, but for some reasons, my agent does not learn very well. I build a custom environment using Unity ML Agents. The environment is very simple: an agent can ...

neural-networks reinforcement-learning actor-critic

asked Jun 20 '19 at 4:41

koryakinp

1993 silver badges10 bronze badges

1

vote

0answers

32 views

Natural actor critic with nonlinear function approximation

I was wondering, if there was a way to implement natural actor critic without having to deal with choosing the right features for the action-state-space. Basically, in Sutton,Barto - Reinforcement ...

reinforcement-learning gradient-descent actor-critic

asked Jun 15 '19 at 12:36

GreenLogic

1836 bronze badges

2

votes

1answer

255 views

Understanding the temporal difference prediction error formula which uses a derivative

I'm very new to understanding the concept of prediction error underlying the output of the critic in the critic-actor method (RL learning), so bear with me, please. For the temporal difference ...

reinforcement-learning actor-critic

asked Feb 11 '19 at 1:25

Hanna Haponenko

313 bronze badges

4

votes

1answer

1k views

Reinforcement Learning - What is the logic behind actor-critic methods? Why use a critic?

Following David Silver's course, I came across the actor-critic policy improvement algorithm family. It holds For one-step Markov decision processes that $$\nabla_{\theta}J(\theta) = \mathbb{E}_{\...

machine-learning neural-networks reinforcement-learning policy-gradient actor-critic

asked Dec 3 '18 at 17:35

Gulzar

2612 silver badges12 bronze badges

1

vote

1answer

3k views

A2C Loss Function Explosion

I am training OpenAI's implementation of the A2C algorithm found here. Based on the mean episode reward graph below we can see it is in fact learning the policy function up until roughly 2000 updates:...

machine-learning neural-networks reinforcement-learning actor-critic

asked May 15 '18 at 21:08

sccrthlt

1501 silver badge8 bronze badges

2

votes

1answer

876 views

What are the differences between contextual bandits, actor-citric methods, and continuous reinforcement learning?

Let's imagine we have a blackbox function f(X) -> y which we don't know. X is a vector of 10 continuous variables, which we ...

reinforcement-learning q-learning multiarmed-bandit actor-critic

asked Jan 8 '18 at 22:22

freesoul

1313 bronze badges

11

votes

1answer

7k views

Actor-critic loss function in reinforcement learning

In actor-critic learning for reinforcement learning, I understand you have an "actor" which is deciding the action to take, and a "critic" that then evaluates those actions, however, I'm confused on ...

machine-learning reinforcement-learning actor-critic

asked Jan 2 '18 at 19:17

tryingtolearn

4895 silver badges11 bronze badges

Stack Exchange Network

current community

your communities

more stack exchange communities

Questions tagged [actor-critic]

Deep deterministic policy gradient : which network do I have to use for testing?

Understanding the Neural Network Output using a Custom Gym Env continuous boundaries

Why is my actor loss reversed?

How to define number of states in reinforcement learning

Why the approximation of $\log \pi_{\theta}(a|s)$ improves numerical stability?

Can't use replay memory with policy gradient, why?

A2C in TensorFlow 2 using model with two heads

Advantage term in A2C

entropy coefficient in A3C [closed]

Reinforcement Learning: A2C agent does not learn

Natural actor critic with nonlinear function approximation

Understanding the temporal difference prediction error formula which uses a derivative

Reinforcement Learning - What is the logic behind actor-critic methods? Why use a critic?

A2C Loss Function Explosion

What are the differences between contextual bandits, actor-citric methods, and continuous reinforcement learning?

Actor-critic loss function in reinforcement learning

Hot Network Questions

Questions tagged [actor-critic]

Related Tags