Questions tagged [temporal-difference]

The tag has no usage guidance.

10 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
4
votes
1answer
430 views

How is TD(1) of TD(lambda) equivalent to Monte Carlo?

In Sutton and Barto's book about RL they say that the TD($\lambda$) algorithm is equivalent to Monte Carlo when $\lambda = 1$. I don't see how that is the case. They define the lambda return as: $$...
1
vote
0answers
34 views

How to analyse temporal variation in R---vigilance of birds

I am currently doing a project for my Bachelor degree, for which I study vigilance in birds. Obviously, I have to analyse the data statistically. However, I do not know where to begin since I do not ...
1
vote
0answers
549 views

In $TD(\lambda)$, is there a way to choose an effective $\lambda$ based on the environment?

In $TD(\lambda)$, is there a way to choose an effective $\lambda$ based on the environment?
0
votes
1answer
77 views

What is temporal leakage?

So I've been trying to work out exactly what temporal leakage is for a while now and I'm getting nowhere. I'm not necessarily looking to code or anything, I'm more so interested in what it actually is ...
0
votes
1answer
28 views

How to normalise changes that occur over time spans of different duration?

I want to compare changes in a variable that occur over time spans of different duration. Here is a hypothetical example: Precipitation in Region A decreased by 300 mm (from 1000 mm to 700 mm) over a ...
0
votes
0answers
38 views

Best RL methods for environments with short episodes

I have a game where the length of an episode is fixed and small - 26 steps. The reward is assigned only once, at the end of the episode. State size has around 350 features; there are at maximum 52 ...
0
votes
0answers
13 views

Does varying maximum achievable reward influence learning for policy gradient methods?

An agent has to go around a map by following a path composed of N checkpoints. The state is the relative position of the next 4 checkpoints. The agent receives +1 every time it takes a checkpoint, and ...
0
votes
0answers
116 views

Reinforcement Learning by Sutton: Episodic and Continuing Tasks

Referred to Reinforcement Learning by Sutton: What are Episodic and Continuing Tasks? In my opinion: $\textbf{Episodic Tasks}$ A task is episodic, if there exists a final time step $T$ so that the ...
0
votes
1answer
31 views

Confusion about the derivation of the TD-Learning update rule

I am currently trying to understand the paper "Learning to Predict by the Methods of Temporal Differences" by Sutton. I am stuck with the following step: (From "Learning to Predict by the Methods of ...
0
votes
0answers
60 views

Adding additional constrains to OpenAi Gym

I'm currently working trough some examples which should finally end in a DQN Reinforcement Learning for the CartPole example in the openAI-Gym. Copied some code from GitHub which isn't deep yet: <...