# Questions tagged [temporal-difference]

The tag has no usage guidance.

10 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
430 views

### How is TD(1) of TD(lambda) equivalent to Monte Carlo?

In Sutton and Barto's book about RL they say that the TD($\lambda$) algorithm is equivalent to Monte Carlo when $\lambda = 1$. I don't see how that is the case. They define the lambda return as: ...
34 views

### How to analyse temporal variation in R---vigilance of birds

I am currently doing a project for my Bachelor degree, for which I study vigilance in birds. Obviously, I have to analyse the data statistically. However, I do not know where to begin since I do not ...
549 views

### In $TD(\lambda)$, is there a way to choose an effective $\lambda$ based on the environment?

In $TD(\lambda)$, is there a way to choose an effective $\lambda$ based on the environment?
77 views

### What is temporal leakage?

So I've been trying to work out exactly what temporal leakage is for a while now and I'm getting nowhere. I'm not necessarily looking to code or anything, I'm more so interested in what it actually is ...
28 views

### How to normalise changes that occur over time spans of different duration?

I want to compare changes in a variable that occur over time spans of different duration. Here is a hypothetical example: Precipitation in Region A decreased by 300 mm (from 1000 mm to 700 mm) over a ...
38 views

### Best RL methods for environments with short episodes

I have a game where the length of an episode is fixed and small - 26 steps. The reward is assigned only once, at the end of the episode. State size has around 350 features; there are at maximum 52 ...
13 views

### Does varying maximum achievable reward influence learning for policy gradient methods?

An agent has to go around a map by following a path composed of N checkpoints. The state is the relative position of the next 4 checkpoints. The agent receives +1 every time it takes a checkpoint, and ...
116 views

### Reinforcement Learning by Sutton: Episodic and Continuing Tasks

Referred to Reinforcement Learning by Sutton: What are Episodic and Continuing Tasks? In my opinion: $\textbf{Episodic Tasks}$ A task is episodic, if there exists a final time step $T$ so that the ...