# Questions tagged [temporal-difference]

The tag has no usage guidance.

13 questions
Filter by
Sorted by
Tagged with
7k views

### When are Monte Carlo methods preferred over temporal difference ones?

I've been doing a lot of research about Reinforcement Learning lately. I followed Sutton & Barto's Reinforcement Learning: An Introduction for most of this. I know what Markov Decision Processes ...
2k views

### Why do temporal difference (TD) methods have lower variance than Monte Carlo methods?

In many reinforcement learning papers, it is stated that for estimating the value function, one of the advantages of using temporal difference methods over the Monte Carlo methods is that they have a ...
89 views

### Compare means between samples, while controlling for sampling differences - valid to use regression this way?

There are two independent samples of people, drawn from a population of a city at times $t_1$ and $t_2$, a decade apart*. The people were asked rate their preference regarding some question $Q$ on a ...
15 views

### How to analyse temporal variation in R---vigilance of birds

I am currently doing a project for my Bachelor degree, for which I study vigilance in birds. Obviously, I have to analyse the data statistically. However, I do not know where to begin since I do not ...
169 views

### How is TD(1) of TD(lambda) equivalent to Monte Carlo?

In Sutton and Barto's book about RL they say that the TD($\lambda$) algorithm is equivalent to Monte Carlo when $\lambda = 1$. I don't see how that is the case. They define the lambda return as: ...
527 views

### In $TD(\lambda)$, is there a way to choose an effective $\lambda$ based on the environment?

In $TD(\lambda)$, is there a way to choose an effective $\lambda$ based on the environment?
8 views

### Does varying maximum achievable reward influence learning for policy gradient methods?

An agent has to go around a map by following a path composed of N checkpoints. The state is the relative position of the next 4 checkpoints. The agent receives +1 every time it takes a checkpoint, and ...
20 views

### Model Checking : Safety and Liveness properties [closed]

I know what Safety and Liveness properties are and the relation between Safety and Badprefixes of a LT property. I wanted to understand about the closure properties and why closure of a safety ...
19 views

18 views

### Understanding distributional Temporal Difference Learning

I am trying to understand a recently published Deepmind Paper. In the supplementary information of the paper (accessible here), the authors explain distributional temporal difference learning. In ...