The tag has no usage guidance.

48 questions
Filter by
Sorted by
Tagged with
1 vote
75 views

### Batches in policy gradient methods – theory vs practice

I am currently trying to understand the implementation of batching in policy gradient / actor-critic methods. My understanding is that these methods in principal work as follows: collect a batch of $N$...
• 11
12 views

### In reinforce algorithm (policy gradient), why is the reward not dependent upon the policy parameters?

The following is the precursor to the policy gradient algorithm Why is the return $r(\tau)$ over the trajectory taken out of the gradient as if it is constant w.r.t the policy parameters $\theta$. Is ...
• 2,553
44 views

### Policy Gradients Therom for Episodic Cases: How are different formulas related?

While I am generally aware of how the Policy Gradients algorithms work theoretically, I was recently a bit confused between two definitions of the Policy Gradient and later the derivation of the ...
• 2,298
1 vote
36 views

I'm looking at two proofs for the policy gradient theorem. van heeswijk starts of with expressing the policy gradient as a gradient over expected reward trajectories: and the rest is easy. However ...
• 3,340
1 vote
35 views

### REINFORCE implementation - should we multiply $G_t$ by $\gamma^t$?

I think I see a contradiction between two blog posts, please help me decide if there's an error somewhere or how do these align: lilian weng claims this: while wouter van heeswijk claims this: [ who ...
• 3,340
106 views

• 123
1 vote
676 views

### REINFORCE algorithm, help for the proof of the variance reduction by subtracting a baseline

I'm trying to find a proof or an approximate argument justifying that, in the REINFORCE algorithm, subtracting a baseline to the episode reward reduces the variance. I believe this proof can be done ...
50 views

### Non authorized actions with Policy gradient algorithm

I am using Reinforce algorithm to learn the optimal policy in problem where the action space is dynamic. It depends on the state, in each state, the agent is allowed to choose among a subset of the ...
• 21
443 views

• 583